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Abstract 


The  four  experiments  reported  in  this  paper  were  conducted  in  support  of  the  COMmand 
Decision  Aiding  Technology  (COMDAT)  (llbg)  Technology  Demonstrator  Project  (TDP)  and 
the  Halifax  Class  Modernization  Command  and  Control  System  (HMCCS)  programme.  The  first 
experiment  assessed  the  relative  visibility  of  the  basic  Naval  Tactical  Data  System  (NTDS)  and 
MIL-STD-2525B  (2525B)  tactical  symbols.  Performance  with  the  colour-coded  versions  of  the 
two  symbol  sets  was  not  significantly  different.  However,  the  air  and  subsurface  symbols  were 
less  discriminable  than  the  surface  symbols.  Recommendations  for  improving  the  discrimination 
of  the  different  warfare  area  symbols  are  included. 

One  potential  advantage  of  the  2525B  symbols  is  the  possibility  of  adding  additional  information 
about  the  track  platform  to  the  basic  symbol  shape.  The  remaining  experiments  assessed  the 
visibility  of  the  basic  symbols  with  iconic  information  added  and  the  visibility  of  the  icons 
themselves.  Adding  iconic  information  did  not  have  a  large  effect  on  the  efficiency  with  which 
the  basic  symbols  were  located  except  when  the  icon  shape  replaced  the  symbol  shape. 
Performance  in  locating  and  recognizing  individual  icons  depended  on  their  complexity  and  their 
uniqueness.  It  was  recommended  that  only  a  small  number  of  highly  discriminable  icons  be  used 
at  any  one  time.  The  use  of  icons  without  the  symbol  frame  should  be  restricted  to  non  tactical 
information  and  such  icons  should  probably  not  be  colour  coded.  Further  research  is  required  to 
determine  how  to  implement  these  recommendations. 
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Resume 


Les  quatre  experiences  dont  il  est  fait  etat  dans  le  present  rapport  ont  ete  menees  a  l’appui  du 
Projet  de  Demonstrateur  de  la  Technologie  (PDT)  d’Aide  aux  Decisions  de  Commandement 
(COMDAT)  (llbg)  et  du  Programme  de  modernisation  du  systeme  de  commandement  et  de 
controle  pour  la  classe  Halifax  ( Halifax  Class  Modernization  Command  and  Control  System 
Programme,  HMCCS).  Dans  le  cadre  de  la  premiere  experience,  on  a  evalue  la  visibilite  relative 
des  symboles  tactiques  de  base  du  Systeme  de  donnees  navales  tactiques  (SDNT)  et  de  la 
M1LSTAND-2525B  (2525B).  Le  rendement  des  versions  codees  en  couleurs  des  deux  jeux  de 
symboles  n’etait  pas  tres  different.  Cependant,  les  symboles  pour  les  entites  aeriennes  et  sub- 
superficielles  etaient  moins  distinguables  que  ceux  utilises  pour  les  entites  de  surface.  Des 
recommandations  pour  1’ amelioration  de  la  discrimination  des  differents  symboles  utilises  dans 
les  zones  de  guerre  sont  foumies. 

L’un  des  avantages  du  jeu  de  symboles  2525B  est  d’offrir  la  possibilite  d’ajouter  a  la  forme  de 
base  du  symbole  de  P information  sur  la  plate-forme  de  poursuite  utilisee.  Les  autres  experiences 
portaient  sur  P  evaluation  de  la  visibilite  des  symboles  de  base  lorsque  de  l’information  iconique 
est  ajoutee  et  sur  la  visibilite  des  icones  elles-memes.  L’ajout  d’information  iconique  n’avait  pas 
un  effet  important  sur  Pefficacite  avec  laquelle  les  symboles  de  base  etaient  localises,  sauf 
lorsque  la  forme  de  l’icone  remplasait  la  forme  du  symbole.  Le  rendement  en  termes  de 
localisation  et  de  reconnaissance  d’ icones  individuelles  dependait  de  leur  complexite  et  de  leur 
unicite.  II  est  recommande  de  n’utiliser  simultanement  qu’un  petit  nombre  d’icones  hautement 
discriminables.  L’utilisation  d’icones  sans  le  cadre  du  symbole  devrait  etre  limitee  uniquement  a 
P  information  non  tactique  et  ces  icones  ne  devraient  probablement  pas  comporter  un 
codage  en  couleurs.  D’autres  recherches  sont  necessaires  afm  de  determiner  comment  mettre  en 
oeuvre  ces  recommandations. 
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Executive  summary 


Evaluation  of  symbol  sets  for  naval  tactical  displays 

Sharon  McFadden;  Jennifer  Jeon;  Annie  Li;  Annalisa  Minniti;  DRDC  Toronto 
TR  2007-046;  Defence  R&D  Canada  -  Toronto;  January  2008. 

Introduction  or  background:  The  research  reported  in  this  paper  was  conducted  under  the 
COMmand  Decision  Aiding  Technology  (COMDAT)  (llbg)  Technology  Demonstrator  Project 
(TDP)  in  support  of  the  Halifax  Class  Modernization  Command  and  Control  System  (HMCCS) 
programme.  One  of  the  goals  for  the  COMDAT  TDP  is  to  develop  improved  Operator  Machine 
Interface  (OMI)  guidelines.  This  includes  selecting  suitable  symbol  sets  for  naval  tactical 
displays.  In  support  of  this  work,  four  experiments  were  carried  out  to  evaluate  some  of  the 
design  concepts  for  symbology  laid  out  in  MIL-STD-2525B.  The  first  experiment  compared  the 
basic  MIL-STD-2525B  (2525B)  tactical  symbols  with  the  Naval  Tactical  Data  System  (NTDS) 
currently  used  on  tactical  displays  in  the  HALIFAX  class  frigates.  The  remaining  experiments 
evaluated  some  of  the  icons  specified  in  MIL-STD-2525B  for  representing  platform  information. 
The  second  experiment  assessed  discrimination  of  the  basic  symbol  shapes,  with  and  without  the 
addition  of  icons,  against  a  plain  and  a  complex  background  while  the  third  experiment  assessed 
discrimination  of  the  icons  themselves.  In  the  fourth  experiment,  participants  searched  for  a 
specific  target  shape  and  then  identified  the  icon  superimposed  on  the  basic  symbol. 

Results:  The  results  of  the  first  experiment  indicated  that  the  colour  coded  2525B  symbol  set 
offered  no  advantage  over  a  well  designed  NTDS  symbol  set  if  both  were  presented  against  a 
plain  dark  grey  background.  With  both  sets,  the  air  and  subsurface  symbols  were  less 
discriminable  than  the  surface  symbols.  A  comparison  of  monochrome  versions  of  the  two 
symbol  sets  did  indicated  that  the  NTDS  shapes  are  more  discriminable  than  the  2525B  shapes. 
However,  visual  search  performance  was  similar  with  the  colour-coded  versions  of  the  two 
symbol  sets.  In  addition,  iconic  information  can  be  added  more  easily  to  the  solid  shapes.  Adding 
iconic  information  did  not  have  a  large  effect  on  the  effectiveness  with  which  the  basic  symbols 
were  located  except  when  the  icon  shape  replaced  the  symbol  shape.  Performance  in  locating  and 
recognizing  individual  icons  depended  on  their  complexity  and  their  uniqueness.  It  was 
recommended  that  only  a  small  number  of  highly  discriminable  icons  be  used  at  any  one  time. 
The  use  of  icons  by  themselves  should  be  restricted  to  non  tactical  information  and  probably 
should  not  be  colour  coded. 

Significance:  The  results  of  these  experiments  will  be  used  to  provide  recommendation  to  the 
Navy  on  the  selection  of  suitable  symbology  for  naval  tactical  displays. 

Future  plans:  It  is  recommended  that  a  more  discriminable  method  of  representing  a  neutral 
contact  be  developed.  The  MIL-STD-2525B  symbols  for  these  types  of  contacts  are  unsuitable 
for  use  on  electronic  displays  when  presented  at  the  recommended  visual  angle.  Further  research 
is  required  to  evaluate  proposed  methods  for  improving  discrimination  between  the  air  and 
subsurface  symbols  and  to  develop  better  guidelines  for  ensuring  the  effective  discrimination  of 
tactical  (foreground)  information  and  contextual  (background)  information. 
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Sommaire 


Evaluation  of  symbol  sets  for  naval  tactical  displays 

Sharon  McFadden;  Jennifer  Jeon;  Annie  Li;  Annalisa  Minniti;  DRDC  Toronto 

TR  2007-046;  R  &  D  pour  la  defense  Canada  -  Toronto;  Janvier  2008. 

Introduction  ou  contexte  :  Les  recherches  ici  presentees  ont  ete  menees  dans  le  cadre  du  Projet 
de  Demonstrateur  de  la  Technologie  (PDT)  d’Aide  aux  Decisions  de  Commandement 
(COMDAT)  (llbg)  a  l’appui  du  Programme  de  modernisation  du  systeme  de  commandement  et 
de  controle  pour  la  classe  Halifax  ( Halifax  Class  Modernization  Command  and  Control  System 
Programme,  HMCCS).  L’un  des  objectifs  du  PDT  du  COMDAT  est  l’elaboration  de  lignes 
directrices  ameliorees  pour  P interface  operateur-machine  (IOM).  Cela  englobe  la  selection  de 
jeux  de  symboles  appropries  pour  les  affichages  navals  tactiques.  Quatre  experiences  ont  ete 
menees  a  l’appui  de  ces  travaux  afm  d’evaluer  certains  concepts  de  symbolisation  exposes  dans  la 
M1LSTAND  2525B.  Dans  la  premiere  de  ces  experiences,  les  symboles  tactiques  de  base  de  la 
M1LSTAND  2525B  (2525B)  ont  ete  compares  a  ceux  du  Systeme  de  donnees  navales  tactiques 
(SDNT)  actuellement  utilises  dans  les  affichages  pour  les  ffegates  de  la  classe  Halifax.  Dans  le 
cadre  des  autres  experiences  on  a  evalue  certaines  des  icones  dont  Tutilisation  est  preconisee  dans 
la  Mil, ST  AND  2525B  pour  la  representation  de  P  information  sur  les  plates-formes.  Dans  la 
deuxieme  experience,  on  a  evalue  la  discrimination  des  formes  de  base  des  symboles  avec  et  sans 
l’ajout  d’icones  lorsqu’ils  figurent  contre  des  arriere-plans  simples  et  complexes;  lors  de  la 
troisieme  experience  on  a  evalue  la  discrimination  des  icones  elles-memes.  Pour  la  quatrieme 
experience,  les  participants  recherchaient  une  forme  de  cible  specifique,  puis  identifiaient  l’icone 
superposee  au  symbole  de  base. 

Resultats  :  Les  resultats  de  la  premiere  experience  indiquent  que  le  jeu  de  symboles  2525B  codes 
en  couleurs  n’offfe  aucun  avantage  par  rapport  a  un  jeu  de  symboles  SDNT  bien  consu  lorsque 
les  deux  sont  affiches  contre  un  arriere-plan  gris  fonce  simple.  Dans  le  cas  des  deux  jeux,  les 
symboles  pour  les  entites  aeriennes  et  sub-superficielles  etaient  moins  discriminables  que  les 
symboles  representant  des  entites  de  surface.  Une  comparaison  des  versions  monochromes  des 
deux  jeux  de  symboles  indique  que  les  formes  des  symboles  du  SDNT  sont  plus  facilement 
discriminables  que  celles  du  2525B.  Cependant,  pour  les  versions  codees  en  couleurs  des  deux 
jeux  de  symboles,  le  rendement  de  la  recherche  visuelle  etait  similaire.  En  outre,  Pinformation 
iconique  peut  etre  plus  facilement  ajoutee  aux  formes  pleines.  L’ajout  d’information  iconique 
n’avait  pas  un  effet  important  sur  Pefficacite  avec  laquelle  les  symboles  de  base  etaient  localises, 
sauf  lorsque  la  forme  de  l’icone  rempla^ait  celle  du  symbole.  Le  rendement  en  termes  de 
localisation  et  de  reconnaissance  d’icones  individuelles  dependait  de  leur  complexity  et  de  leur 
unicite.  II  est  recommande  de  n’utiliser  simultanement  qu’un  petit  nombre  d’icones  hautement 
discriminables.  L’utilisation  d’icones  seules  devrait  etre  limitee  uniquement  a  Pinformation  non 
tactique  et  ces  icones  ne  devraient  probablement  pas  comporter  un  codage  en  couleurs. 

Portee  :  Les  resultats  de  ces  experiences  serviront  a  faire  des  recommandations  a  la  Marine  quant 
au  choix  du  jeu  de  symboles  convenant  le  mieux  aux  affichages  tactiques  navals. 

Recherches  futures  :  II  est  recommande  d’elaborer  une  methode  de  representation  des  contacts 
neutres  permettant  une  meilleure  discrimination.  Les  symboles  de  la  MILSTAND  2525B  pour 
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trois  types  de  contacts  ne  conviennent  pas  aux  dispositifs  d’affichage  electronique  lorsque 
presentes  a  Tangle  visuel  recommande.  D’autres  recherches  sont  necessaires  afm  d’evaluer  les 
methodes  proposees  d’ amelioration  de  la  discrimination  des  symboles  utilises  pour  des  entries 
aeriennes  et  sub-superficielles  ainsi  que  pour  elaborer  de  meilleures  lignes  directrices  assurant 
une  discrimination  efficace  entre  l’information  tactique  (avant-plan)  et  l’information  contextuelle 
(arriere-plan). 
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Introduction 


The  purpose  of  the  COMmand  Decision  Aiding  Technology  (COMDAT)  Technology 
Demonstrator  Project  (TDP)  is  to  research  and  demonstrate  Multi-Source  Data  Fusion  (MSDF) 
technologies  and  carry  out  human  factors  studies  to  support  evolutionary  upgrades  to  the  Flalifax 
Class  Command  and  Control  System  (CCS)  in  the  areas  of  battle  space  awareness.  One  of  the 
goals  of  the  human  factors  studies  is  to  develop  improved  guidelines  for  designing  the  Operator 
Machine  Interface  (OMI)  for  naval  command  and  control  systems. 

To  support  the  development  of  improved  OMI  guidelines,  a  COMDAT  OMI  Style  Guide  (Unger 
Campbell  2004)  was  produced  that  distilled  all  of  the  accepted  knowledge  on  the  design  of 
military  tactical  displays  and  adapted  it  for  use  with  the  CCS  and  the  COMDAT  Technology 
Demonstrator.  In  terms  of  symbology,  the  guide  provided  the  designer  with  the  option  of  using 
the  Naval  Tactical  Data  System  (NTDS)  symbol  set  currently  on  the  CCS  or  adopting  either 
North  Atlantic  Treaty  Organisation  (NATO)  draft  STANdardisation  AGreement  (STANAG) 
4420  or  Military  Standard  2525B  (MIL-STD-2525B).  However,  it  cautioned  that  “Without 
further  research  only  those  elements  of  draft  STANAG  4420  or  MIL-STD-2525B  that  correspond 
to  the  current  NTDS  symbols  should  be  used”  (Section  15.1.6).  The  studies  reported  in  this  paper 
contribute  towards  the  research  required  for  improved  guidance  on  the  design  and  use  of  these 
alternative  symbol  sets  for  tactical  displays. 


Background 

Currently,  the  NTDS  symbol  set  is  used  on  Canadian  Forces  (CF)  CCS  displays  (Figure  1).  The 
shape  of  the  symbol  codes  its  affiliation  or  identification  (ID)  (friend-circle,  hostile-diamond, 
unknown-square)  while  environment  or  warfare  area  is  coded  by  the  symbol’s  orientation 
(pointing  up  indicates  air;  pointing  down  indicates  subsurface,  and  full  symbol  indicates  surface). 
With  the  advent  of  colour  coding,  colour  coded  versions  of  these  symbols,  in  which  ID  is 
redundantly  colour  coded,  are  sometimes  used.  As  well,  new  symbols  have  been  added  to 
represent  neutral  targets  and  other  variants  of  the  symbols  created  in  an  effort  to  provide 
additional  or  more  precise  information  to  the  operator  about  a  contact  or  track. 


Figure  1:  Basic  shapes  in  the  Naval  Tactical  Data  System  (NTDS)  symbol  set. 


Most  of  the  additions  were  carried  out  in  an  ad-hoc  manner  as  the  need  arose.  Moreover,  there  is 
no  standard  version  of  the  symbols  especially  across  navies.  Thus  in  the  early  1990’s,  a  NATO 
Working  Group  was  tasked  to  develop  a  standard  symbol  set  for  naval  tactical  displays  (draft 
STANAG  4420)  that  would  take  advantage  of  the  capabilities  of  modem  display 
technology  and  be  consistent  with  human  factor  guidelines  for  displays  (North  Atlantic  Treaty 
Organisation  1992). 
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The  new  shapes  proposed  by  STANAG  4420  were  based  on  the  NTDS  symbology.  In  fact  the 
document  indicated  that  a  modified  form  of  the  NTDS  symbology  might  continue  in  use  for  the 
representation  of  basic  warfare  area  and  ID  information.  To  improve  the  visibility  of  the  NTDS 
symbols  on  raster  scan  displays  they  recommended  the  use  of  two  pixel  wide  lines.  However,  it  is 
not  clear  from  the  draft  document  if  the  updated  NTDS  symbols  would  also  be  colour  coded. 

To  accommodate  additional  platform  information,  it  proposed  solid  versions  of  the  NTDS 
symbols  that  could  be  overlaid  with  iconic  information.  As  well,  the  number  of  ID  categories  that 
could  be  represented  was  increased  to  include  assumed  friend,  etc,  through  the  use  of  a  dashed 
outline  and  a  new  symbol  was  added  for  neutral  tracks.  ID  information  was  redundantly  colour 
coded:  red  for  hostile,  blue  for  friendly,  off-white  for  unknown  and  green  for  neutral.  Military 
platforms  all  had  these  four  basic  shapes  with  amplifying  information  overlaid  as  icons  or  letter 
identifiers.  Non-military  tracks  and  weapons  were  represented  by  icons  alone.  Since  STANAG 
4420  was  never  ratified,  the  complete  symbol  set  was  never  published.  However,  a  U.S. 
evaluation  of  the  proposed  symbology  includes  a  short  description  of  draft  STANAG  4420  and  its 
development  (NAVSPAWARSYSCOM  1991). 

As  a  precursor  to  ratification,  the  U.S.  carried  out  a  relatively  extensive  evaluation  of  the 
proposed  design  concepts  (Kirkpatrick,  Dutra,  Lyons,  Osga,  and  Puggi  1992b, 
NAVSPAWARSYSCOM  1991).  In  it  they  compared  the  NATO  symbols  with  the  basic  NTDS 
symbol  set  that  was  currently  in  use,  a  colour  coded  version  of  the  NTDS  symbol  set  with 
information  tags,  and  a  colour  coded  outline  version  of  the  NATO  symbols.  The  information  tags 
provided  the  same  information  as  the  icons  in  the  NATO  set.  This  allowed  the  investigation  of  the 
merits  of  design  concepts  such  as  colour  coding,  filled  shapes,  and  iconic  representation  of 
amplifying  information.  In  the  evaluation,  experienced  operators  carried  out  operationally 
relevant  tasks  including  a  recognition  task  (e.g.  find  all  hostile  air  tracks),  a  scenario  following 
task  (e.g.  engage  hostile  fighter  approaching  own  ship)  and  a  monitoring  task  (indicate  new 
tracks).  Tasks  were  carried  out  using  a  series  of  different  scenarios  and  a  simulated  tactical 
display.  For  the  recognition  task,  response  times  were  significantly  faster  with  the  NATO 
symbols  than  the  other  three  symbol  sets  and  accuracy  was  significantly  better  than  with  the 
monochrome  NTDS  symbols.  In  the  scenario  following  task,  participants  tended  to  perform  better 
with  the  NATO  symbols,  but  not  significantly  so.  The  use  of  operational  tasks  and  participants 
who  were  familiar  with  NTDS  symbology  supported  the  potential  benefit  of  the  design  concepts 
presented  in  draft  STANAG  4420.  However,  the  evaluation  also  pointed  out  limitations  and 
recommended  certain  changes.  For  example,  they  found  that  it  was  impossible  to  produce  dashed 
outlines  that  were  clearly  visible.  They  also  recommended  more  systematic  evaluations  of  factors 
such  as  symbol  and  icon  discrimination,  clutter,  symbol  size,  and  colour  contrast. 

Some  of  the  evaluations  proposed  above  were  carried  out  as  part  of  the  same  overall  project. 
Separate  studies  looked  at  the  effect  of  contrast,  luminance,  and  saturation  (Van  Orden,  Osga,  and 
Lauben  1991,  Van  Orden  and  Benoit  1994),  and  symbol  size  (Laxar  and  Van  Orden  1994). 
Although  some  results  favoured  the  NATO  symbols  (Van  Orden  et  al.  1991),  with  others  there 
were  no  differences  (Van  Orden  and  Benoit  1994)  and  in  at  least  one  case  the  colour  coded 
NTDS  symbols  proved  superior  (Laxar  and  Van  Orden  1994).  Since  the  studies  varied  in  their 
methodology  and  conditions  tested,  it  is  difficult  to  draw  definitive  conclusions.  However,  overall 
there  is  some  evidence  that  superior  performance  with  the  NATO  symbols  was  due  in  part  to  the 
use  of  a  grey  background.  In  all  of  the  studies  where  the  NATO  symbols  proved  superior  or 
equivalent  to  the  NTDS  colour  coded  symbols,  the  NATO  symbols  were  presented  against  a 
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medium  grey  background  and  the  NTDS  symbols  were  presented  against  a  dark  or  saturated 
background.  The  grey  background  would  tend  to  make  the  NATO  symbols  appear  more  saturated 
(e.g.  redder)  while  the  dark  and  saturated  backgrounds  would  make  the  NTDS  symbols  appear 
less  saturated  (McFadden,  Kaufmann,  and  Janzen  1994,  Ware  and  Cowan  1982).  When  highly 
saturated  NTDS  symbols  were  used,  the  differences  between  the  two  symbol  sets  disappeared. 

The  symbol  recognition  and  scenario  following  task  made  use  of  some  of  the  icons  available  in 
NATO  4420,  but  there  was  no  systematic  evaluation  of  them.  Another  follow-up  to  the  original 
evaluation  (Kirkpatrick,  Dutra,  Heasly,  Granda,  and  Vingelis  1992a)  focused  specifically  on  icon 
recognition.  Seventy-four  symbols  composed  of  all  the  unique  icons  in  the  surface,  subsurface, 
and  air  categories  along  with  selected  special  point  and  miscellaneous  symbols  were  tested. 
Participants  were  shown  74  sets  of  ten  symbols  composed  of  a  target  symbol  and  nine  distractor 
symbols.  The  distractor  symbols  were  selected  from  the  test  set  and  other  special  point  symbols. 
The  symbol  sets  were  presented  on  a  CRT  with  the  name  of  the  target  symbol  to  be  located  at  the 
top  of  the  screen.  Accuracy  and  response  times  were  collected  and  confusion  matrices  generated. 
Symbols  with  zero  errors  tended  to  use  two  letter  labels  rather  than  icons  although  this  was  not 
universally  true.  Exceptions  usually  arose  because  the  two  letter  code  was  inconsistent  with  the 
user  experience.  Icons  that  physically  resembled  the  object  also  tended  to  have  lower  error  rates. 
Unfortunately,  the  visual  angle  of  the  symbols  was  not  indicated.  Thus,  it  is  not  possible  to 
determine  if  the  symbol  sizes  were  consistent  with  recommendations  for  tactical  displays1. 

Around  the  same  time  that  NATO  was  developing  its  standard  for  naval  tactical  displays,  the  US 
was  working  on  a  common  warfighting  symbology  standard  to  eliminate  conflicts  across  the  three 
environments.  Initially,  they  based  the  design  on  the  U.S.  Army  symbology  set,  but  as  a  result  of 
the  evaluation  of  the  draft  NATO  STANAG  4420  symbology,  they  adopted  many  of  its  design 
concepts.  The  main  difference  was  the  use  of  a  yellow  club  or  clover  shape  for  unknown  tracks 
instead  of  the  square.  A  green  square  shape  was  used  for  neutral  tracks.  The  most  recent  version 
of  this  standard  is  MIL-STD-2525B  with  Change  2  (Department  of  Defence  2006).  To  the  best  of 
our  knowledge,  no  systematic  evaluation  of  the  2525B  symbols  has  been  carried  out  beyond  the 
development  of  recommendations  for  implementing  the  standard  (NAVSEA  Dahlgren  2006). 

Overall,  it  would  appear  that  the  design  concepts  proposed  in  draft  STANAG  4420  and 
incorporated  into  MIL-STD-2525B  have  merit.  Operators  experienced  with  the  monochrome 
NTDS  symbology  set  found  the  4420  symbols  easy  to  learn  and  use.  There  was  little  evidence  of 
negative  transfer  from  the  NTDS  symbols  (NAVSPAWARSYSCOM  1991).  In  subjective 
evaluations,  operators  responded  positively  to  the  4420  symbology  feeling  that  the  colour  coding 
and  the  availability  of  amplifying  information  on  the  symbol  would  be  helpful.  However,  it  was 
also  evident  from  the  results  that  the  proposed  symbols  might  not  provide  the  anticipated  level  of 
improvement,  especially  if  the  alternative  was  a  colour  coded  NTDS  symbology  set.  Secondly, 
none  of  the  studies  evaluated  all  of  the  basic  symbols  in  the  two  sets.  Most  of  them  used  only  the 
air  symbols  as  targets  and  in  some  cases  subsurface  symbols  were  not  even  present  on  the 
display.  With  the  move  towards  joint  operations  this  is  a  serious  deficiency.  Earlier  research  with 
monochrome  and  colour-coded  NTDS  symbols  had  found  that  the  air  and  sub-surface  symbols 
were  less  discriminable  from  each  other  than  from  the  surface  symbols  (Jacobsen,  Neri,  and 


1  According  to  MIL-STD-2525B,  tactical  symbols  should  have  a  visual  angle  of  between  30-40  minutes. 
However,  they  do  not  specify  a  viewing  distance  since  that  would  be  system  dependent.  Since  draft 
STANAG  4420  was  never  promulgated,  its  recommended  viewing  angle  is  not  known. 
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Rogers  1985).  Moreover,  there  were  problems  with  the  icon  set  (Kirkpatrick  et  al.  1992a)  and  the 
use  of  a  dashed  outline  (NAVSPAWARSYSCOM  1991).  The  MIL-STD  2525B  symbol  set  also 
differs  from  the  recommendation  of  4420  to  use  solid  versions  of  the  NTDS  symbol  set  to  avoid 
negative  transfer.  The  basic  shape  for  the  unknown  symbol  is  a  clover  rather  than  a  square.  They 
use  the  square  symbol  for  neutral  tracks  (Figure  2)  and  the  different  warfare  areas  are 
differentiated  by  whether  or  not  the  outline  around  the  symbols  is  missing  from  the  top  of  the 
square  (subsurface),  the  bottom  (air)  or  is  complete  (surface).  Given  the  visibility  problem  with 
dashed  lines,  this  is  a  potential  concern.  The  4420  proposal  for  unknown  tracks  indented  the 
square  at  the  bottom  for  air  tracks  and  at  the  top  for  sub-surface  tracks.  Their  neutral 
symbol  was  also  a  square  with  the  indentation  replaced  by  an  equivalent  protrusion.  Since  the 
2525B  symbology  has  never  been  systematically  evaluated,  the  impact  of  these  design  decisions 
is  not  known. 


Figure  2:  MIL-STD-2525B  air,  surface  and  subsurface  neutral  symbols 


Current  research 

The  proposed  research  attempts  to  understand  the  relative  strengths  and  weaknesses  of  the  NTDS 
and  2525B  symbol  sets  in  order  to  recommend  improvements  to  the  design  concepts  laid  out  in 
MIL-STD-2525B.  While  it  is  clear  from  the  previously  cited  studies  that  colour  coding  improves 
the  visibility  or  discrimination  of  both  symbol  sets,  it  is  not  clear  that  the  solid  shapes  are  more 
conspicuous  than  the  outline  shapes  especially  if  saturated  colours  and  optimal  backgrounds  are 
used  with  the  NTDS  symbols  (Laxar  and  Van  Orden  1994,  Van  Orden  and  Benoit  1994).  As  well, 
all  of  the  studies  cited  used  the  NTDS  symbols  with  single  pixel  wide  lines.  As  noted  in  draft 
STANAG  4420,  the  original  NTDS  symbols  are  potentially  less  conspicuous  with  raster  scan 
displays  as  display  resolution  increases.  The  problem  will  likely  get  worse  as  high  resolution 
LCD  displays  are  introduced  into  military  displays  (Hollands,  Parker,  McFadden,  and  Boothby 
2002).  Thus,  4420  recommended  using  thicker  lines.  The  relative  ineffectiveness  of  the  NTDS 
symbols  when  coded  with  less  saturated  colours  is  further  evidence  that  the  continued  use  of 
single  pixel  wide  lines  is  not  advisable.  A  potential  problem  with  both  symbol  sets  is  the  result 
reported  above  that  the  air  and  sub-surface  symbols  are  less  discriminable  from  each  other  than 
from  the  surface  symbols.  Moreover,  in  the  studies  by  Van  Order  and  Benoit  (1994),  the  hostile 
symbol  tended  to  be  less  visible  than  the  friendly  symbol.  Given  that  hostile  air  symbols  should 
be  at  least  as  conspicuous  as  all  the  other  symbols,  it  is  important  to  understand  the  extent  and 
possibly  the  source  of  the  above  results. 

For  all  of  the  above  reasons,  Experiment  1  assessed  the  relative  visibility  of  each  of  the  nine  basic 
NTDS  and  2525B  shapes  (see  Table  1  in  the  method  section)  with  and  without  the  benefit  of 
colour  coding  using  a  visual  search  task.  Initially,  we  had  planned  to  evaluate  the  12  basic  2525B 
symbols.  In  pilot  studies,  it  proved  virtually  impossible  to  discriminate  the  air,  surface,  and  sub¬ 
surface  neutral  symbols  (Figure  2).  It  was  very  difficult  to  determine  whether  or  not  a  horizontal 
line  was  present  at  the  top  or  bottom  of  the  square  when  the  symbols  were  presented  at  the 
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recommended  visual  angle  on  a  high  resolution  display.  Thus,  the  evaluation  of  the  2525B 
symbols  was  restricted  to  the  nine  basic  symbols  that  corresponded  to  the  basic  NTDS  set.  As 
well,  our  evaluations  were  based  on  the  symbol  set  as  promulgated  in  MIL-STD-2525B 
(Department  of  Defence  1999).  Recent  changes  to  this  standard  were  released  in  2005  and  2006 
after  most  of  these  studies  had  been  completed.  However,  none  of  the  changes  in  the  revised 
standard  resulted  in  significant  changes  to  the  symbols  and  icons  evaluated  in  these  studies. 

In  keeping  with  the  recommendations  of  4420,  we  constructed  NTDS  symbols  with  thicker  lines. 
This  also  made  it  more  reasonable  to  use  the  same  chromaticities  and  background  with  both 
symbol  sets.  As  Van  Orden  and  Benoit  (1994)  point  out,  with  moderately  desaturated  colours,  it 
is  possible  to  increase  symbol  luminance  which  improves  contrast  and  visibility.  It  also  allows 
more  flexibility  in  the  choice  of  colours. 

Performance  was  assessed  using  a  visual  search  task.  Visual  search  is  an  important  component  of 
the  tasks  carried  out  by  operators  monitoring  tactical  displays.  Moreover,  there  is  a  rich  literature 
on  the  factors  affecting  visual  search  performance  since  it  has  been  used  extensively  to  measure 
performance  in  simple  and  complex  displays  (e.g.Teichner  and  Krebs  1972,  Treisman  and  Gelade 
1980,  Wolfe,  Cave,  and  Frannzel  1989).  A  counting  task  was  used  over  the  more  common  yes/no 
task,  because  we  have  found  that  it  provides  a  better  estimate  of  the  discriminability  of  different 
symbols  (McFadden,  Bauer,  and  McManus  1997).  Time  to  locate  multiple  targets  improved 
dramatically  when  the  coding  method  supported  the  grouping  of  like  objects  across  a  spatial 
extent.  Moreover,  operators  need  to  determine  where  the  hostile  contacts  as  well  as  if  a  hostile 
contact  is  present.  Both  requirements  are  represented  in  the  counting  task. 

In  a  visual  search  task,  participants  usually  search  for  a  specific  object  (called  the  target)  in  a 
background  composed  of  one  or  more  ‘distractor’  objects.  The  distractors  can  vary  both  in 
number  present  (e.g.  five)  and  in  number  of  types  (e.g.  number  of  colours).  In  these  experiments, 
there  were  always  50  objects  on  the  display  since  we  were  primarily  interested  in  evaluating 
performance  on  a  cluttered  display  as  opposed  to  looking  at  the  effect  of  number  of  objects  on 
visibility.  Performance  was  measured  when  all  the  distractors  were  the  same  (one-distractor-type 
condition)  and  when  distractors  included  all  of  the  eight  non-target  symbols  in  the  symbol  set 
(2525B  or  NTDS)  to  which  the  target  symbol  belonged  (eight-distractor-type  condition).  Thus, 
participants  searched  for  instances  of  one  the  nine  basic  symbols,  in  either  the  2525B  or  NTDS 
symbol  sets,  presented  in  a  background  composed  of  all  remaining  symbols  in  its  set.  The  eight- 
distractor-type  condition  provides  direct  information  on  the  relative  visibility  of  the  different 
symbols  on  a  cluttered  display.  The  results  for  the  one-distractor-type  condition  were  subjected  to 
multi-dimensional  scaling  to  assess  the  relative  visibility  of  the  members  of  each  set  and  to 
understand  the  dimensions  people  are  using  to  discriminate  amongst  the  various  symbols 
(McFadden  et  al.  1997,  Tomonaga  and  Matsuzawa  1992). 

One  potential  advantage  of  the  2525B  symbol  set  is  the  possibility  of  adding  more  information 
about  the  underlying  contact  to  the  basic  symbol  shape.  As  with  draft  NATO  STANAG  4420, 
2525B  specifies  a  large  number  of  icons  and  modifiers  that  can  be  added  to  the  basic  symbol 
shapes  for  this  purpose.  In  the  initial  evaluation  of  the  4420  symbols  (NAVSPAWARSYSCOM 
1991),  operators  responded  positively  to  the  augmented  symbols  because  they  reduced  the 
requirement  to  hook  or  select  the  contact  in  order  to  get  additional  information  about  platform 
type.  However,  the  icon  shapes  are  often  more  complex  than  the  basic  symbol  shapes  and,  since 
they  fit  inside  the  symbol,  subtend  a  smaller  visual  angle.  The  same  study  expressed  concern  that 
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the  operator’s  ability  to  discriminate  or  even  recognize  some  symbols  could  be  affected  by  factors 
such  as  display  type  and  resolution  and  symbol  size.  Despite  these  concerns,  there  has  been  little 
direct  evaluation  of  them  beyond  the  follow-on  study  by  Kirkpatrick  et  al.  (1992a).  It  did  not 
address  the  issue  of  visibility  or  increased  display  complexity  with  the  augmented  symbols. 

Thus,  Experiments  2  through  4  were  carried  out  to  gather  additional  data  on  the  impact  of 
augmenting  the  basic  2525B  symbology  with  platform  information.  A  total  of  27  icons  (9  air,  9 
surface,  and  9  subsurface),  that  are  representative  of  the  range  of  available  platform  types  and  that 
are  likely  to  be  encountered  relatively  frequently  in  CF  naval  operations,  were  used  in  these 
experiments.  The  27  icons  represent  only  a  subset  of  the  available  platform  types  in  MIL-STD 
2525B.  It  would  be  very  difficult  to  assess  all  of  the  icons  and  it  is  unlikely  that  all  of  them  would 
be  used  in  a  naval  context.  The  icons  selected  are  typical  of  the  platform  types  that  could  be 
encountered  relatively  frequently  by  the  Halifax  Class  frigate.  An  additional  criterion  was  that  the 
icons  represent  the  different  ways  that  platform  information  is  represented  in  2525B.  Thus,  the 
chosen  symbols  (see  Table  4  in  Experiment  2)  include  iconic  representations  of  a  platform  type 
(e.g.  fixed  winged  aircraft,  nuclear  submarine),  alphanumeric  representation  of  a  platform  type, 
(e.g.  BB  for  battleship),  icon  plus  alphanumerics  (e.g.  rotary  wing  attack  helicopter),  and  no 
frame  (e.g.  mine,  decoy). 

Experiment  2  assessed  the  impact  of  adding  iconic  information  to  the  2525B  symbology.  The  task 
was  identical  to  the  eight- distractor  condition  in  Experiment  1.  Participants  searched  for  examples 
of  one  of  the  basic  colour-coded  2525B  symbol  shapes  in  a  background  composed  of  the 
remaining  eight  symbols.  The  displayed  symbols  were  either  the  basic  set  used  in  Experiment  1 
or  the  augmented  set  described  in  the  previous  paragraph. 

Graphical  icons  are  only  one  method  of  providing  an  operator  with  more  easily  accessible 
information  about  the  tactical  situation.  Contextual  information  can  be  added  to  the  background 
as  well.  This  can  range  from  adding  graphical  information  such  as  range  rings  or  air  corridors, 
sensor  information  such  as  radar  paints,  and/or  geographical  information  in  the  form  of  digital  or 
vector  maps.  This  contextual  information  may  also  impact  the  visibility  of  the  tactical  information 
about  a  track’s  ID  and  warfare  area.  To  assess  the  potential  negative  impact  of  such  contextual 
information  on  the  visibility  of  the  tactical  symbols.  Experiment  2  also  compared  performance 
with  the  basic  and  augmented  symbols  against  a  plain  and  a  complex  background. 

Experiment  3  examined  the  relative  visibility  of  the  different  augmented  symbols.  Participants 
had  to  search  for  a  particular  platform  type  in  a  background  of  the  other  nine  platforms  for  that 
warfare  area  (the  basic  symbol  was  one  of  the  alternatives).  All  of  the  symbols  presented 
simultaneously  had  the  same  warfare  area  and  ID  or  basic  shape.  It  is  unlikely  that  operators 
would  want  to  search  for  a  specific  platform  (e.g.  locate  all  the  helicopters).  However,  the  more 
efficiently  they  are  able  to  locate  a  specific  icon,  the  more  likely  they  will  quickly  recognize  it 
when  scanning  a  cluttered  display.  By  understanding  the  characteristics  of  icons  that  are  located 
efficiently,  we  can  develop  better  guidelines  for  designing  them. 

Experiment  4  examined  the  participants’  ability  to  classify  the  platform  type  on  a  cluttered 
display.  A  good  icon  should  be  both  easily  discriminated  and  intuitively  interpreted.  They  were 
required  to  search  for  one  of  the  basic  symbol  shapes  and  then  identify  its  platform  type.  Since 
the  participants  were  not  initially  familiar  with  the  icons,  they  were  given  extensive  training  and 
provided  with  an  information  sheet  that  showed  each  icon  and  its  label. 
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Experiment  1  -  Comparison  of  NTDS  and  2525B  symbol 
sets 


Method 

Experiment  1  examined  visual  search  performance  with  four  symbol  sets  -  monochrome  NTDS, 
colour-coded  NTDS,  monochrome  2525B  symbols,  colour-coded  2525B  symbols.  Participants 
counted  the  number  of  target  symbols  in  a  background  of  non- target  symbols  (distractors).  The 
distractors  were  either  all  the  non-target  symbols  in  the  target  symbol’s  set  or  one  of  the  non¬ 
target  symbols. 


Participants 

Ten  participants  (5  males  and  5  females)  took  part.  Their  ages  ranged  from  21  to  50  with  a  mean 
of  32.8.  All  participants  had  normal  or  corrected-to-normal  vision  (based  on  self-report)  and 
normal  colour  vision  as  assessed  by  the  Ishihara  Colour  Plates  (38  plate  edition).  Each 
participant  signed  an  informed  consent  form  approved  by  the  DRDC  Human  Research  Ethics 
Committee  (HREC)  before  participating  in  the  experiments  (DRDC  HREC  Protocol  L483). 

Conditions 

Experiment  1  was  a  4  (NTDS-monochrome,  NTDS-colour,  2525B-monochrome,  2525B-colour 
symbol  sets)  by  9  (symbol  label)  by  2  (one-  or  eight-distractor-type)  design.  Each  symbol  set 
consisted  of  9  different  symbols:  3  warfare  areas  (air,  surface,  subsurface)  by  3  different  ID 
(friendly,  hostile,  unknown).  Each  target  symbol  was  presented  in  a  background  composed  of 
each  of  the  other  remaining  symbols  in  its  set  for  a  total  of  72  runs  in  each  one-distractor-type 
condition  (9  target  symbols  by  8  non-target  symbols)  and  in  a  background  composed  of  all  the 
remaining  symbols  for  a  total  of  9  runs  for  each  eight-distractor-type  condition.  Thus,  each 
participant  completed  324  runs  (81  runs  by  4  symbol  sets).  Half  the  participants  started  with  one 
of  the  NTDS  symbol  sets  and  half  with  one  of  the  2525B  sets.  As  well,  half  of  the  participants 
started  with  a  monochrome  and  half  with  a  colour  coded  symbol  set.  Participants  always 
completed  both  monochrome  and  colour-  coded  versions  of  a  symbol  set  before  moving 
on  to  the  other  set. 


Apparatus 

Stimuli  were  presented  on  a  19”  diagonal  Hitachi  Superscan  814  Monitor  set  to  1280  by  1024 
pixels.  All  responses  were  collected  via  the  numeric  keypad  on  the  computer  keyboard.  The 
chromaticity  and  luminance  of  each  of  the  stimuli  were  measured  with  a  Minolta  CS100  chroma 
meter.  The  illuminance  of  the  screen  and  keyboard  were  measured  using  a  Hagner  photometer  in 
illuminance  mode. 
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Stimuli 


The  stimuli  were  modified  versions  of  the  9  basic  NTDS  symbols  and  the  equivalent  2525B 
symbols  (Table  1).  In  both  sets,  geometric  shape  is  used  to  code  ID  (diamond  -  hostile,  square 
(NTDS)  or  club  (2525B)  -  unknown,  and  circle  -  friendly)  and  whether  the  shape  is  closed 
(surface)  cut  off  at  the  bottom  (air)  or  at  the  top  (subsurface)  is  used  to  code  warfare  area.  The 
visual  angle  of  each  shape  is  also  shown  in  Table  1.  The  symbol  sets  differ  primarily  in  that 
NTDS  uses  outlines  and  2525B  uses  filled  shapes.  The  relative  size  of  the  symbols  was  based  on 
the  recommendations  in  MIL-STD-2525B  for  that  symbol  set  and  observation  of  operational 
displays  for  the  NTDS  symbol  set. 

The  NTDS  symbols  were  created  using  Corel  Draw  and  saved  in  Windows  MetaFile  (WMF) 
format  for  importation  into  the  experimental  control  program.  The  2525B  symbols  were  available 
in  Corel  Draw  file  format.  Every  effort  was  taken  to  make  the  symbols  for  each  warfare  area  (e.g. 
the  circle,  square,  and  diamond)  the  same  size.  The  small  differences  shown  in  Table  1  probably 
arose  from  display  and  software  limitations. 

Based  on  the  results  of  pilot  studies,  some  modifications  were  made  to  each  symbol  set.  The  lines 
that  make  up  each  NTDS  symbol  were  made  thicker  for  the  reasons  discussed  in  the  introduction. 
The  main  modification  to  the  2525B  symbols  was  the  use  of  a  completely  closed  outline  for  all 
symbols.  The  standard  calls  for  no  outline  at  the  bottom  of  the  air  symbols  and  at  the  top  of  the 
subsurface  symbols.  However,  as  discussed,  this  additional  cue  to  warfare  area  was  not  visible  in 
the  pilot  studies.  A  closed  outline  was  also  consistent  with  NATO  Draft  STANAG  4420  (North 
Atlantic  Treaty  Organisation  1992).  In  addition,  the  outline  was  increased  in  thickness.  In  the 
samples  supplied  with  MIL-STD-2525B  (Department  of  Defence  1999),  the  outline  was  not 
visible  on  our  monitors  when  the  symbols  were  reduced  down  to  the  recommended  size. 

In  the  colour  coded  conditions,  ID  was  redundantly  colour  coded.  To  avoid  confounding  the 
effect  of  colour  coding  with  the  effect  of  specific  colours,  the  colours  specified  for  the  2525B 
colour  set  were  used  with  both  colour  sets.  MIL-STD-2525B  did  not  specify  chromaticity 
coordinates  or  RGB  values  for  symbols,  but  it  does  include  a  set  of  sample  symbols.  The 
default  colours  (RGB  values)  for  these  sample  symbols  were  used.  Table  2  shows  the  RGB  values 
for  all  the  stimuli  used  in  Experiment  1  and  the  chromaticity  coordinates  and  luminances 
for  the  Hitachi  monitor. 
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Table  1:  Colour-coded  2525B  and  NTDS  symbol  sets.  The  height  and  width  of  each  symbol  on  the 
display  in  minutes  of  arc  at  a  viewing  distance  of  60  cm  is  shown  below  the  symbol. 


Symbol  Set 

Warfare  Area 

ID  and  Size  (minutes  of  arc) 

Friendly 

Hostile 

Unknown 

Air 

Q 

40  by  40 

a 

40  by  40 

(2) 

40  by  50 

2525B 

Surface 

o 

40  by  40 

o 

50  by  50 

o 

50  by  50 

Subsurface 

u 

40  by  40 

0 

40  by  40 

<3> 

40  by  50 

Air 

20  by  46 

23  by  46 

20  by  40 

NTDS 

Surface 

46  by  46 

50  by  50 

40  by  40 

Subsurface 

20  by  46 

23  by  46 

20  by  40 
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Table  2:  CIE  1931  Chromaticity  coordinates  (x,y)  and  average  luminance  (L  in  Cd/m2 )  of  each  of 
the  symbols  and  backgrounds  for  the  Hitachi  814  monitor. 


Symbol  and  background 
colours 

Gun  values 

Chromaticity 
coordinates  and 
luminances 

Red 

Green 

Blue 

X 

y 

L 

Red  (Hostile) 

255 

128 

128 

0.454 

0.336 

30 

Blue  (friendly) 

128 

227 

255 

0.207 

0.262 

54 

Y ellow  (unknown) 

255 

255 

128 

0.372 

0.467 

83 

Grey  symbol 

223 

223 

222 

0.282 

0.314 

62 

Monochrome  background 

79 

79 

79 

Too  low 

3.2 

Display 

On  each  trial,  participants  were  shown  a  display  containing  50  symbols  randomly  placed  in  the 
cells  of  an  imaginary  8  (vertical)  by  8  (horizontal)  array  presented  against  a  monochrome  dark 
grey  background.  The  symbols  were  randomly  offset  between  0  and  90  minutes  of  arc  in  both  x 
and  y  to  make  the  display  appear  unordered  and  it  usually  resulted  in  a  few  of  the  symbols 
partially  overlapping.  The  nominal  size  of  the  grid  was  approximately  23.5  by  23.5  degrees  of  arc 
at  a  viewing  distance  of  60  cm.  The  50  symbols  were  made  up  of  between  3  and  6  target  symbols 
and  44  and  47  distractor  symbols.  In  the  single  distractor  conditions,  all  of  the  distractor  symbols 
were  identical.  Figure  3  shows  an  example  of  a  single  and  an  eight-distractor-type  display. 


Task 

The  participant’s  task  was  to  count  the  number  of  target  symbols  on  the  display  as  quickly  and 
accurately  as  possible  and  to  indicate  their  response  by  pressing  the  appropriate  number  key. 


10 


DRDC  Toronto  TR  2007-046 


□ 

] 

0 

> 

0 

c 

n 

o 

r\ 

A 

V 

A 

V 

A 

□  O  v 

□ 

L-,  ^ 

□ 

n 

V/ 

s y 

LU 

r\ 

1—1 

□  „ 

A.  O 

o 

n 

n 

m  a 

G 

rn 

A 

A 

V 

0 

0 

c 

LU 

Figure  3:  Examples  of  the  displays  presented  to  the  participants.  The  top  image  shows  a  typical 
display  in  the  2525B,  one-distr actor-type,  monochrome  condition  and  the  bottom  image  shows  a 
typical  display  in  the  NTDS,  eight-distractor-type,  colour-coded  condition. 
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Procedure 


The  whole  experiment  took  a  total  of  ten  sessions,  carried  out  on  different  days,  with  each  session 
taking  approximately  90  minutes  including  breaks  to  complete.  Prior  to  starting  the  experiment, 
the  participant  read  the  general  information  sheet  and  signed  the  informed  consent  form.  Any 
questions  regarding  the  general  design  of  the  experiment  was  answered.  Next  the  participant’s 
colour  vision  was  tested  using  the  Ishihara  Pseudoisochromatic  colour  plates.  The  participants 
then  completed  a  set  of  practice  runs  to  familiarize  them  with  the  task,  the  stimulus 
configurations,  and  the  symbols.  The  practice  sessions  were  identical  to  the  test  sessions  except 
that  only  a  small  number  of  runs  and  trials  per  run  were  carried  out. 

The  experiment  was  carried  out  under  relatively  low  ambient  illumination  to  maximize  the 
visibility  of  the  symbols.  The  light  falling  on  the  screen  was  approximately  7  Lux  and  the  light 
falling  on  the  keyboard  16  Lux.  Participants  were  seated  in  an  adjustable  chair  in  front  of  an 
adjustable  keyboard.  The  nominal  viewing  distance  when  the  participants  were  seated  properly  in 
the  chair  was  60  cm.  However,  viewing  distance  was  not  controlled  and  participants  were 
free  to  adjust  their  viewing  distance  if  they  wished.  The  viewing  distance  of  operators  using 
tactical  displays  would  not  be  controlled  and  it  was  not  required  for  the  experimental  conditions 
being  manipulated. 

At  the  start  of  each  run,  a  screen  specifying  the  target  and  distractor  symbols  for  that  run  was 
presented.  This  screen  remained  on  until  the  participant  signalled  with  a  keystroke  to  continue. 
On  each  of  the  324  runs,  4  practice  trials  were  presented  (one  with  each  of  the  different  numbers 
of  targets)  followed  by  12  experimental  trials.  The  practice  trials  gave  the  participants  a  chance  to 
become  familiar  with  the  target  symbol  for  that  run.  The  display  for  each  trial  remained  on  the 
screen  until  a  response  was  made.  Feedback  in  the  form  of  a  plus  for  a  correct  response  or  a 
minus  for  an  incorrect  response  was  given  after  each  trial.  The  participant’s  response  and  the  time 
taken  to  make  that  response  were  recorded.  Participants  were  instructed  to  enter  their  count  using 
the  number  keypad  on  the  right  of  the  keyboard,  and  to  make  their  responses  as  quickly  and 
accurately  as  possible.  Also,  they  were  advised  not  to  lean  forward  but  to  sit  comfortably  and  to 
keep  the  distance  between  the  screen  and  their  eyes  relatively  consistent.  They  were  also 
encouraged  to  take  breaks  between  runs  especially  if  they  felt  fatigued.  At  the  end  of  every  six 
runs,  a  screen  was  displayed  telling  the  participants  to  take  a  break.  However,  participants  were 
free  to  take  a  break  after  each  run. 

The  order  of  presentation  of  the  different  targets  within  a  condition  and  the  order  of  the 
conditions  within  a  symbol  set  were  randomized  across  participants. 

Statistical  analysis 

The  results  of  interest  were  median  response  time  per  run  for  correct  responses  and  counting 
accuracy  per  run.  The  median  response  time  per  run  was  used  to  control  for  outliers.  Although 
response  times  were  relatively  normally  distributed  (usually  skewness  was  less  than  1.5  across 
conditions),  using  log  response  times  did  reduce  skewness  to  less  than  1  across  all  conditions. 
Thus  log  response  times  were  used  in  all  the  analyses  reported  in  the  paper.  Prior  to  conducting 
the  main  analysis,  the  effect  of  number  of  targets  on  response  time  was  analysed  to  determine 
whether  response  time  per  trial  or  per  target  was  the  most  accurate  measure.  The  results  indicated 
that  response  time  did  not  increase  as  number  of  targets  increased.  Thus,  response  time  per  trial 
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was  used  in  all  subsequent  analyses.  Since  there  is  often  a  trade  off  between  response  time  and 
accuracy  in  visual  search  tasks,  multivariate  analyses  of  median  log  response  times  and  accuracy 
were  also  carried  out.  The  Wilks’  Lambda  statistic  is  reported  for  the  multivariate  analysis.  For 
all  statistical  tests,  a  significance  level  of  0.01  was  used.  The  Scheffe  test  was  used  for  post- hoc 
analysis  of  the  univariate  analyses. 


Results 

In  terms  of  performance,  our  primary  interest  was  in  the  data  from  the  eight-distractor-type 
conditions.  It  is  unlikely  that  operators  would  face  a  display  with  only  two  types  of  symbols.  The 
one-distractor-type  condition  was  carried  out  to  better  understand  the  dimensions  participants 
were  using  to  discriminate  amongst  the  different  symbols  in  each  set.  Thus,  the  two  distractor- 
type  conditions  were  analysed  separately.  Initially,  a  repeated  measures  analysis  of  variance  was 
conducted  on  the  eight-distractor-type  data  to  determine  the  effect  of  symbol  set.  This  was 
followed  by  separate  analyses  of  the  effect  of  symbol  shape  on  performance  for  each  of  the 
symbols  sets.  Multi-dimensional  scaling  was  used  on  the  one-distractor-type  data  to  examine  the 
possible  dimensions  used  in  searching  for  the  different  targets  within  each  symbol  set. 

Effect  of  symbol  set  and  symbol  shape  on  performance 

On  average,  there  was  a  significant  effect  of  symbol  set  on  accuracy,  F(3,27)  =  33.1,  and  response 
time,  F(3,27)  =  71.9  and  overall  F(6,52)  =  40.3.  A  post-hoc  analysis  indicated  that  accuracy  was 
significantly  better  with  the  colour  coded  NTDS  symbols  relative  to  the  two  monochrome  symbol 
sets  (Figure  4).  In  addition  performance  with  the  colour-coded  2525B  symbol  set  was 
significantly  better  than  the  2525B  monochrome  symbol  set.  Response  times  with  both  colour 
coded  symbols  set  were  significantly  faster  than  with  the  monochrome  symbols  and  response 
times  with  the  monochrome  NTDS  symbols  were  significantly  faster  than  with  the  monochrome 
2525B  symbols. 
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Figure  4:  Percent  errors  and  response  times  for  each  of  the  symbols  sets  in  the  eight-distractor- 
type  condition.  The  standard  error  bars  are  also  shown  in  this  and  subsequent  figures. 

The  results  of  the  analyses  of  variance  for  symbol  shape  for  each  symbol  set  are  shown  in  Table 
3.  As  can  be  seen,  differences  in  accuracy  were  found  only  with  the  monochrome  symbol  sets. 
Post-hoc  analysis  of  the  effect  of  symbol  shape  on  accuracy  for  the  monochrome  symbols  showed 
that  the  surface  friendly  was  significantly  more  accurate  than  the  subsurface  and  air  hostile 
symbols  for  the  2525B  symbols  and  the  surface  hostile  for  the  NTDS  symbols. 
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Table  3:  Summary >  of  repeated  measures  analyses  of  variance  of  symbol  for  each  symbol  set. 
Results  for  accuracy,  response  time,  and  a  multivariate  analysis  of  both  measures  are  shown. 
Column  2  shows  the  univariate  followed  by  the  multivariate  degrees  of freedom. 


Symbol  set 

Univariate  / 
multivariate 
degrees  of  freedom 

F  values  (p  <  0.01) 

Accuracy 

Response  time 

Multi-variate 

Monochrome  NTDS 

8,72/16,142 

9.9 

19.6 

21.6 

Monochrome  2525B 

8,72/16,142 

7.2 

20.2 

10.3 

Colour-coded  NTDS 

8,72/16,142 

n.s* 

46.8 

23.3 

Colour-coded  2525B 

8,72/16,142 

n.s 

12.8 

8.2 

*Not  significant. 


Symbol  shape  had  a  significant  effect  on  response  time  and  accuracy  and  response  time  combined 
for  all  four  symbol  sets.  Post-hoc  analyses  of  the  effect  of  symbol  shape  on  response  time  showed 
a  similar  trend  to  the  accuracy  results.  As  can  be  seen  in  Figure  5,  response  times  were  always 
shortest  for  the  surface  friendly.  With  the  monochrome  2525B  symbols,  response  times  were  also 
shorter  for  the  unknown  symbols.  However,  with  the  colour-coded  2525B  symbols,  the 
subsurface  and  air  unknown  symbols  had  significantly  longer  response  times  relative  to  the 
surface  friendly  and  the  surface  unknown.  With  the  colour  coded  NTDS,  response  times  for  the 
three  surface  symbols  were  significantly  faster  than  all  the  remaining  symbols.  The  trend  was 
similar  for  the  monochrome  symbols  except  for  the  surface  hostile  which  had  a  significantly 
longer  response  time  than  the  other  surface  symbols.  Overall,  the  effect  of  symbol  shape  is  very 
similar  for  the  two  colour  coded  symbol  sets.  The  primary  difference  is  that  response  times  for 
the  surface  symbols  appear  to  be  somewhat  faster  with  the  NTDS  set. 
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Figure  5:  Average  response  time  for  a)  monochrome  and  b)  colour  coded  symbols  with  each  of 

the  symbols  in  each  symbol  set. 


Relative  visibility  of  each  symbol  within  the  four  symbol  sets 

The  primary  reason  for  including  the  one-distractor-type  condition  was  to  obtain  a  better 
understanding  of  how  the  choice  of  symbol  shapes  influenced  discrimination  of  each  symbol  in 
the  eight-distractor-type  condition.  As  a  first  step,  we  carried  out  a  Spearman  rank  order 
correlation  for  each  symbol  set  to  estimate  the  correlation  between  response  times  in  the  one-  and 
eight-distractor-type  conditions.  In  the  one-distractor  type  condition,  response  times  were 
averaged  across  all  the  runs  with  different  distractor  symbols  for  each  target  symbol.  On  average, 
we  would  expect  the  rank-order  of  response  times  for  the  symbols  in  the  eight-distractor-type 
condition  to  be  similar  to  the  rank  order  of  the  average  response  time  in  the  one-distractor-type 
condition  if  participants  were  using  the  same  dimensions  to  find  the  target  symbols  under  both 
conditions.  Spearman  rank  order  correlations  ranged  from  0.82  in  the  monochrome  conditions  to 
0.9  with  the  colour-coded  symbol  sets. 
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Next,  the  underlying  features  that  could  have  been  used  by  the  participants  in  the  different  symbol 
sets  were  examined  using  multidimensional  scaling.  A  weighted  Euclidean  model  was  fit  to  the 
response  times  matrices  for  each  symbol  set  using  the  MDS  procedure  in  SAS"  (SAS 
Institute  Inc.  1992).  The  response  times  were  treated  as  ratio  data  and  a  single  partition  was  used. 
The  analyses  were  carried  out  on  a  square  (rather  than  a  triangular)  matrix  because  the  matrices 
were  asymmetrical. 

Based  on  the  design  of  the  symbols,  a  two  dimensional  fit  might  be  expected.  One  dimension 
would  be  ID  which  is  represented  by  the  basic  geometric  shape  (and  colour  for  the  colour  coded 
symbol  sets)  and  the  second  dimension  would  be  warfare  area  represented  by  whether  the  shape 
is  whole  or  cut  off  at  the  top  or  bottom.  Following  the  recommendation  of  Davison  (Davison 
1983),  a  separate  solution  was  calculated  for  1  to  6  dimensions.  The  MDS  procedure  provides 
several  measures  of  the  fit  of  the  data  to  a  particular  solution.  For  a  weighted  Euclidean  model, 
either  the  badness-of-fit  criterion  (or  stress  value)  or  the  correlation  between  the  data  and  the 
transformed  distances  can  be  used  (Davison  1983).  With  the  first  measure,  the  lower  the  value  is, 
the  better  the  fit;  with  the  second,  the  higher  the  value,  the  better  the  fit.  Figure  6  shows  the 
values  determined  for  these  two  criteria  as  a  function  of  the  dimensionally  of  the  solution  for  each 
symbol  set. 

The  standard  method  for  selecting  the  appropriate  solution  is  a  clear  elbow  or  knee  in  the  plot  of 
fit  versus  dimensionality  of  the  solution.  The  absence  of  a  clear  elbow  and  a  relatively  high 
correlation  indicates  a  one  dimensional  solution.  As  can  be  seen  in  Figure  6,  a  three  dimensional 
fit  would  seem  to  be  most  appropriate  for  these  data. 


Number  of  Dimensions 


Figure  6:  Two  measure  of fit  for  each  symbol  set  for  one  to  six  dimension  MDS  solution.  A 
straight  line  has  been  fit  by  eye  to  the  higher  dimension  fit  measures. 

The  three  dimensional  solutions  for  the  four  symbol  sets  are  shown  in  Figure  7  (monochrome 
symbols)  and  Figure  8  colour-coded  symbols).  In  each  case,  the  graph  on  the  left  shows 
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Dimensions  1  and  2  and  the  graph  on  the  right  Dimensions  1  and  3.  Since  the  solutions  are 
always  constrained  to  fall  within  about  plus  or  minus  two,  distances  cannot  be  compared  for  the 
different  symbol  sets. 
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Figure  7:  Three  dimensional  multidimensional  scaling  solutions  for  the  monochrome  NTDS  (top) 

and  2525B  (bottom)  symbol  sets. 


For  the  monochrome  NTDS  symbols,  Dimension  1  appears  to  differentiate  the  surface  (whole 
shapes)  from  the  remaining  shapes.  The  separation  between  the  remaining  two  warfare  areas  is 
relatively  small  but  consistent  along  that  dimension.  Dimension  2  differentiates  shapes  with 
prominent  right  angles  (hostile  plus  surface-unknown)  from  the  remaining  symbols  and 
Dimension  3  differentiates  the  unknown  shapes  plus  the  subsurface  hostile  from  the  other 
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symbols.  Of  specific  interest  is  the  fact  that  the  surface  hostile  is  confused  with  both  the  surface- 
friendly  and  surface-unknown  symbols  in  Dimension  1 ,  with  the  surface-unknown  in  Dimension 
2  and  the  surface-friendly  in  Dimension  3.  This  would  account  for  the  relatively  high  response 
times  for  this  symbol  in  Figure  5. 

For  the  monochrome  2525B  symbols,  Dimension  1  differentiates  the  unknown  symbols  from  the 
remaining  symbols.  The  remaining  two  dimensions  do  not  appear  as  obvious  as  with  the  NTDS 
symbols  even  though  the  goodness  of  fit  is  similar.  Certainly  the  dimensions  do  not  correlate  well 
with  either  ID  or  warfare  area.  Dimension  2  could  be  seen  as  differentiating  the  surface  friendly 
and  surface-unknown  (circular  shapes)  from  the  remaining  symbols  while  dimension  3 
differentiates  the  surface  hostile  (the  only  symbol  with  strong  angles)  from  the  remaining  shapes. 
The  overall  pattern  of  results  suggests  that  the  air-friendly,  air-hostile,  subsurface-friendly  and 
subsurface-hostile  could  not  be  easily  discriminated  from  each  other.  This  is  certainly  consistent 
with  the  results  of  Figure  5. 

The  MDS  solution  for  the  NTDS  colour  symbols  (Figure  8)  is  very  similar  to  that  for  the 
monochrome  NTDS  symbols  except  that  the  dimensions  are  more  clearly  defined  and  the  order  of 
the  dimensions  has  changed.  Now,  Dimension  1  differentiates  the  unknown  from  the  friendly  and 
hostile  symbols  and  Dimension  3  the  friendly  symbols  from  the  other  two.  Together,  the  symbols 
are  clearly  differentiated  based  on  their  colour  coding  and  basic  shape  over  Dimensions  1  and  3. 
Dimension  2  differentiates  the  symbols  along  warfare  area  lines.  However,  the  order  of  the 
unknown  air  and  subsurface  symbols  is  reversed  and  they  are  relatively  close  together  compared 
to  the  friendly  and  hostile  symbols.  This  relative  difference  is  consistent  with  the  relatively  poor 
performance  of  the  air  and  subsurface  symbols  in  Figure  5.  As  with  the  solutions  in  Figure  7, 
symbols  with  the  same  orientation  do  not  necessarily  group  together. 

The  solution  for  the  2525B  symbols  is  similar  to  that  for  the  NTDS  colour  symbols  except  that 
Dimensions  2  and  3  are  switched  and  symbols  with  the  same  colour  are  more  tightly  grouped. 
This  would  suggest  that  discrimination  within  a  colour  is  poorer  relative  to  the  NTDS  symbols. 
On  the  other  hand,  the  pattern  in  Dimension  3  is  consistent  with  warfare  area.  The  similarities 
between  the  hostile  and  friendly  symbols  for  air  and  subsurface  have  been  overcome  through  the 
use  of  colour  coding. 
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Figure  8:  Three  dimensional  multidimensional  scaling  solutions  for  the  colour  coded  NTDS  (top) 

and  2525B  (bottom)  symbol  sets. 


Discussion 

The  purpose  of  Experiment  1  was  to  conduct  a  more  systematic  comparison  of  the  NTDS  and 
2525B  symbol  sets  and  to  improve  our  understanding  of  the  reasons  for  any  differences  in 
visibility.  Thus,  counting  accuracy  and  response  time  for  each  of  the  9  basic  symbol  shapes  in  the 
two  symbol  sets  was  measured  in  a  background  composed  of  all  the  other  symbols  in  the  set.  In 
addition,  response  time  and  accuracy  for  each  symbol  when  only  one  type  of  distractor  symbol 
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was  present  was  measured  for  each  possible  pair  of  symbols  within  each  set.  The  latter  data  were 
subjected  to  multidimensional  scaling  in  order  to  determine  the  dimensions  people  used  to 
discriminate  amongst  the  symbols. 

As  indicated  in  Figure  4,  overall  performance  was  very  similar  on  the  colour  coded  NTDS  and 
2525B  symbols  sets.  Thus,  when  the  symbols  were  presented  on  the  same  background,  and  the 
recommended  form  (North  Atlantic  Treaty  Organisation  1992)  for  the  NTDS  symbols  was  used, 
the  2525B  symbols  showed  no  advantage  over  NTDS  symbols.  The  use  of  the  same  background 
and  thicker  lines  probably  contributed  to  the  visibility  of  the  NTDS  symbols.  On  the  other  hand, 
there  would  be  no  significant  disadvantage  in  moving  from  a  colour-coded  NTDS  to  the 
equivalent  2525B  symbol  set. 

Both  symbols  sets  have  problems  that  were  not  identified  in  previous  comparisons  of  the  NTDS 
and  NATO  symbols  (Kirkpatrick  et  al.  1992a,  Laxar  and  Van  Orden  1994, 
NAVSPAWARSYSCOM  1991)  which  primarily  evaluated  the  air  and  surface  symbols.  When  all 
nine  symbols  are  presented  simultaneously,  performance  with  the  air  and  subsurface  symbols  is 
considerably  poorer  than  with  the  surface  symbols.  An  examination  of  the  multidimensional 
scaling  solutions  gives  some  insight  into  why  this  happens.  The  air  and  subsurface  symbols  are 
not  clearly  separated  on  any  of  the  dimensions  in  Figure  7  or  Figure  8.  Most  of  the  dimensions 
support  the  discrimination  of  the  different  ID  dimensions.  The  exceptions  are  Dimension  1  in  the 
solution  for  the  monochrome  NTDS  symbols,  Dimension  2  for  the  colour-coded  NTDS  symbols, 
and  Dimension  3  for  the  colour-coded  2525B  symbols.  They  separate  out  the  surface  symbols 
from  the  air  and  subsurface  symbols.  In  the  colour  coded  solutions,  Dimension  2  (NTDS)  and  3 
(2525)  also  tends  to  separate  out  the  air  and  subsurface,  but  the  distances  are  smaller  and  less 
stable  suggesting  that  orientation  is  not  a  strong  cue. 

The  use  of  colour  coding  does  improve  discrimination  of  the  symbols.  A  comparison  of  the 
monochrome  and  colour  coded  solutions  indicates  that  colour  coding  reduces  confusion  across 
the  different  ID  shapes.  This  is  especially  true  for  the  2525B  symbols.  In  the  MDS  solution  for 
the  2525B  monochrome  symbols,  none  of  the  dimensions  discriminate  between  the  air-friendly 
and  air-hostile  and  the  subsurface-friendly  and  subsurface-hostile  symbols.  The  small  differences 
in  the  shape  of  the  four  symbols  (air-hostile,  air-friendly,  subsurface-hostile,  and  subsurface- 
friendly)  are  not  particularly  conspicuous  when  scanning  a  cluttered  display. 

The  superior  performance  with  colour  coding  is  consistent  with  previous  studies  comparing  the 
NTDS  and  NATO  4420  symbols  (NAVSPAWARSYSCOM  1991).  It  is  hypothesized  that  people 
are  able  to  use  colour  to  reduce  the  search  space  (Hollands  et  al.  2002,  Keillor,  Thompson, 
Smallman,  and  Cowen  2003).  This  phenomenon  is  often  referred  to  as  pre-attentive  processing 
(Treisman  1982).  The  result  for  the  monochrome  symbols  suggests  that  symbol  shape  does  not 
tend  to  be  as  effective  in  this  regard.  However,  the  superior  performance  with  the  surface  friendly 
symbol  (circle  shape)  across  all  four  symbol  sets  and  the  unknown  symbols  in  the  2525B 
monochrome  symbol  set  indicates  that  shape  can  be  effective  if  it  is  sufficiently  unique. 
Moreover,  the  MDS  analysis  of  the  monochrome  symbols  suggests  that  size  is  an  effective  coding 
dimension.  The  difference  between  the  surface  and  non-surface  symbols  is  more  distinct  with  the 
monochrome  NTDS  symbols  compared  to  the  monochrome  2525B  symbols.  The  larger  size  of 
the  NTDS  surface  symbols  contributes  substantially  to  their  visibility  and  reduces  the  likelihood 
that  they  will  be  confused  with  an  air  or  subsurface  symbol.  Unfortunately,  reducing  the 
size  of  some  of  the  2525B  symbols  would  make  them  less  suitable  for  adding  icons. 
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Overall,  these  results  underline  the  importance  of  making  sure  that  critical  dimensions  can  be 
easily  discriminated. 

There  are  several  ways  that  the  visibility  of  the  air  and  subsurface  symbols  might  be  improved. 
One  possibility  would  be  to  colour  code  warfare  area  as  well  as  ID.  However,  this  would 
potentially  increase  the  complexity  of  the  display.  It  would  also  reduce  the  ability  of  operators  to 
group  targets  on  the  basis  of  ID.  Since  ID  is  the  most  critical  dimension,  every  effort  should  be 
made  to  support  the  operator’s  ability  to  segregate  the  display  along  that  dimension. 

Another  possibility,  suggested  by  Smallman,  St.  John,  Oonk  and  Cowan  (2001),  would  be  to 
remove  the  redundant  shape  coding  for  ID.  Each  of  the  warfare  areas  could  then  be  given  a 
unique  shape.  If  such  a  solution  were  considered,  careful  consideration  would  have  to  be  given  to 
the  choice  of  symbols  to  avoid  interference  with  previous  experience.  As  well,  most  human 
factors  guidelines  still  recommend  the  use  of  redundant  coding  of  colour-coded  objects.  Colour 
discrimination  tends  to  be  more  susceptible  to  adverse  environmental  (bright  sunlight)  and 
technology  (failure  of  one  channel)  effects  and  to  individual  differences  (colour 
deficiency)  than  shape  coding. 

The  method  that  has  been  explored  most  frequently  is  filtering.  In  its  extreme  form,  an  operator, 
monitoring  only  the  air  or  subsurface  picture,  would  have  the  option  of  removing  the  irrelevant 
symbols.  A  less  extreme  form  is  to  use  a  combination  of  the  two  symbol  sets.  Such  a  concept  was 
proposed  and  tested  by  Osga  and  Keating  (1994).  They  investigated  the  utility  of  allowing 
operators  to  use  different  symbol  sets  including  monochrome  and  colour  coded  NTDS  and  colour 
coded  NATO  symbols  to  create  filters  for  different  categories  of  tracks.  Operators  could  define 
their  own  categories  to  support  their  watchkeeping  tasks  and  assign  a  different  symbol  set  to  each 
category.  They  evaluated  the  number  and  types  of  filters  that  operators  created,  their  performance 
in  a  dynamic  scenario  using  the  filters  that  had  been  created,  and  subjective  opinion  of  the 
concept  and  suitability  of  the  symbol  sets.  Most  operators  felt  that  the  availability  of  the  filters 
made  the  tactical  picture  much  clearer  by  improving  visual  contrast  for  different  types  of  tracks. 
In  a  follow-up  study  (Van  Orden,  Nugent,  La  Fleur,  and  Moncho  1999)  participants  searched  for 
specific  symbols  in  a  display  composed  of  symbols  from  either  one  symbols  set  (colour-coded 
NTDS)  or  two  (colour  coded  NTDS  and  NATO  4420)  or  three  (monochrome  NTDS  plus 
colour  coded  NTDS  and  NATO  4420)  different  symbol  sets.  Again,  only  surface  and  air  symbols 
were  used.  The  results  showed  no  real  advantage  to  the  use  of  multiple  symbol  sets. 
However,  the  symbol  sets  were  not  used  to  differentiate  the  symbols  along  a  dimension  of  interest 
such  as  warfare  area. 

A  recent  U.S.  report  (NAVSEA  Dahlgren  2006)  includes  specific  recommendations  for 
implementing  this  concept  using  the  2525B  symbols  set.  They  recommend  that  operators  have  the 
option  of  using  both  solid  and  outline  versions  of  the  2525B  symbols.  The  solid  versions  would 
be  used  for  the  warfare  area  that  is  currently  being  monitored.  This  would  avoid  the  potential 
differences  in  the  basic  unknown  symbol  shape  between  the  NTDS  and  2525B  symbols. 
Although  it  is  not  explicitly  stated,  their  outline  symbols  appear  to  use  only  single  pixel  wide 
lines.  For  the  reasons  discussed  in  the  introduction,  we  would  recommend  the  use  of  thicker  lines. 
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Experiment  2  -  Impact  of  background  and  symbol 
complexity _ 


Method 

Experiment  2  examined  the  effect  of  symbol  complexity  and  display  background  on  performance 
in  a  visual  search  task.  In  this  experiment  only  the  colour  coded  2525B  symbols  were  used.  The 
symbols  were  presented  against  either  a  monochrome  grey  background  or  a  complex  map 
background.  Participants  search  for  a  specified  symbol  with  a  given  ID  and  warfare  area.  The 
symbols  presented  were  either  the  basic  colour-coded  2525B  symbols  used  in  Experiment  1  or  an 
extended  symbol  set  that  also  provided  platform  information. 


Participants 

Nine  participants  (4  males  and  5  females)  took  part  in  Experiment  2.  Their  ages  ranged  from  19 
to  51  with  a  mean  of  26.1.  All  participants  had  normal  or  corrected-to-normal  vision  (based  on 
self-report)  and  normal  colour  vision  as  assessed  by  the  Ishihara  Colour  Plates  (38  plate  edition). 
Each  participant  signed  an  informed  consent  form  approved  by  the  Defence  Research  and 
Development  Canada  Ethics  Committee  before  participating  in  the  experiments  (DRDC  Ethics 
protocol  #L563). 

Conditions 

Experiment  2  was  a  two  (simple  or  complex  background)  by  two  (basic  or  complex  symbols) 
design.  In  the  basic  symbol  conditions,  the  symbols  were  identical  to  those  used  in  the  colour- 
coded  2525B  conditions  in  Experiment  1.  In  the  complex  symbol  conditions,  an  extended  set  of 
2525B  symbols  were  used.  This  extended  set  provided  information  on  platform  type  (e.g. 
battleship,  civilian  aircraft)  as  well  as  ID  and  warfare  area.  The  symbols  (basic  and  complex) 
were  presented  against  either  a  monochrome  grey  background  or  a  complex  map  background. 
Participants  completed  9  runs,  one  with  each  basic  symbol  shape  as  a  target,  in  each  of 
the  4  conditions,  for  a  total  of  36  runs.  The  order  of  the  four  conditions  was  randomized  across 
participants  and  the  order  of  presentation  of  the  9  symbol  shapes  as  target  was  randomized  across 
conditions  and  participants. 


Apparatus 

The  stimuli  were  presented  on  a  19”  (0.48  m)  diagonal  ViewSonic  Professional  Series  P817 
monitor  set  to  1280  by  1024  pixels.  All  responses  were  collected  using  the  computer  keyboard. 
Measurements  of  chromaticity,  luminance,  and  illuminance  were  carried  out  with  the  same 
instruments  as  in  Experiment  1 . 
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Stimuli 


The  stimuli  were  30  hostile,  30  friendly,  and  30  unknown  colour-coded  2525B  symbols.  Table  4 
shows  the  30  hostile  symbols.  The  friendly  and  unknown  symbols  were  identical  to  the  hostile 
symbols  except  for  their  colour  coding  (blue  for  friendly  and  yellow  for  unknown).  As  can  be 
seen,  the  30  symbols  included  10  air,  10  surface,  and  10  subsurface  platform  types.  For  the  most 
part,  platform  type  is  represented  by  an  icon  within  the  basic  symbol.  In  a  few  cases  (mine  and 
decoy),  the  icon  is  presented  without  the  basic  shape  or  frame  as  it  is  referred  to  in 
MIL-STD-2525B.  The  visual  angle  of  the  basic  symbol  was  identical  to  those  used  in 
Experiment  1.  For  clarity  throughout  the  remainder  of  the  report,  the  term  symbol  will  be  used 
when  referring  to  the  basic  symbol  (as  shown  in  Table  1)  or  the  augmented  symbols  (symbol  plus 
icon)  and  the  term  icon  will  be  used  to  refer  to  the  shapes  that  provide  platform  information. 

As  much  as  possible  the  icons  were  presented  as  supplied  by  MIL-STD-2525B.  However,  a  few 
were  modified  because  they  were  very  difficult  to  discriminate  when  presented  on  our  display. 
The  decoy  and  mine  icons  were  increased  in  size  and  outlined  completely  in  a  thicker  black  line 
as  they  were  extremely  difficult  to  see  in  their  original  format.  Also  the  ‘A’  and  ‘K’  in  the  rotary 
wing  attack  and  tanker  icons  ,  the  ‘BB’  for  battleship  and  the  ‘WPN’  for  weapon  were 
increased  slightly  in  size.  The  visual  angles  of  the  basic  symbols  were  identical  to  those  used  in 
Experiment  1  (See  Table  1). 

The  RGB  values  of  the  stimuli  in  Experiment  2  were  identical  to  Experiment  1.  However,  a 
different  monitor  was  used  so  the  chromaticity  coordinates  and  luminance  of  the  stimuli  were 
slightly  different.  Table  5  shows  the  chromaticity  coordinates  for  each  of  the  basic  shapes. 
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Table  4:  The  air,  surface  and  subsurface  hostile  symbols  used  in  Experiment  2. 
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Table  5:  CIE  1931  Chromaticity  coordinates  (x,y)  and  average  luminance  (L  in  Cd/m2 )  of  each  of 
the  symbols  and  backgrounds  on  the  ViewSonic  Professional  Series  P817  monitor. 


Symbol  and  background 
colours 

Gun  values 

Chromaticity 
coordinates  and 
luminances 

Red 

Green 

Blue 

X 

y 

L 

Red  (Hostile) 

255 

128 

128 

0.412 

0.328 

38 

Blue  (friendly) 

128 

227 

255 

0.212 

0.266 

63 

Y ellow  (unknown) 

255 

255 

128 

0.358 

0.444 

89 

Monochrome  background 

192 

192 

192 

0.277 

0.308 

52 

Display 

The  display  was  identical  to  Experiment  1  with  a  few  small  exceptions.  The  symbols  were 
presented  against  either  a  complex  background  (Figure  9)  or  a  monochrome  grey  background 
with  approximately  the  same  average  luminance  as  the  complex  one.  In  the  basic  symbol 
condition,  the  3-6  targets  presented  on  each  display  were  identical.  In  the  complex  symbol 
condition,  the  target  symbols  could  be  any  of  the  10  platform  types  associated  with  that  basic 
symbol  shape.  There  were  always  multiple  types  of  distractors  and  the  distractors  were  the  non¬ 
target  symbol  shapes.  Thus  in  the  basic  symbol  conditions,  the  distractors  were  the  other  8  basic 
symbol  shapes  as  shown  in  Table  1  and  in  the  complex  symbol  conditions,  they  could  be  any  of 
the  remaining  80  symbols  (8  basic  non-target  shapes  by  ten  platform  types). 
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Figure  9:  Map  used  as  the  background  in  Experiment  2. 


Tasks 

The  participant’s  task  was  identical  to  Experiment  1.  They  were  shown  one  of  the  basic  symbol 
shapes  and  required  to  count  the  number  of  occurrences  of  all  symbols  with  that  ID  and  warfare 
area.  Thus,  when  a  basic  subsurface  symbol  was  presented,  participants  were  expected  to  include 
examples  of  the  mine  and  decoy  in  their  count. 


Procedure 

The  procedure  for  Experiment  2  was  identical  to  Experiment  1  with  a  few  exceptions.  Experiment 
2  was  run  in  conjunction  with  Experiments  3  and  4.  The  three  experiments  took  a  total  of  five 
sessions,  carried  out  on  different  days,  with  each  session  taking  approximately  2  hours  including 
breaks  to  complete.  The  order  of  the  four  conditions  in  Experiment  2  was  randomized  across 
participants,  but  each  condition  was  completed  before  going  on  to  the  next.  For  each  of 
the  36  runs,  participants  completed  4  practice  trials  followed  by  16  test  trials.  At  the  end  of  every 
three  runs,  a  screen  was  displayed  telling  the  participants  to  take  a  break.  However,  participants 
were  free  to  take  a  break  after  each  run.  Illuminance  levels  were  approximately  12  Lux  on  the 
screen  and  1 5  Lux  on  the  keyboard. 
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Statistical  analysis 

The  data  preparation  and  basic  statistical  analysis  were  identical  to  Experiment  1. 


Results 

Experiment  2  examined  the  effect  of  adding  iconic  information  (Table  4)  and  a  map  background 
on  discrimination  of  the  2525B  symbols.  Counting  performance  with  the  basic  symbols  presented 
against  a  monochrome  grey  background  was  compared  with  the  same  symbols  presented  against 
a  map  background  and  with  the  complex  symbols  presented  against  both  backgrounds.  In  general, 
counting  performance  was  poorer  with  the  complex  symbols  compared  to  the  basic  symbols  and 
the  effect  was  somewhat  greater  with  a  map  background  compared  to  the  plain  background 
(Figure  10).  As  shown  in  Table  6,  there  was  a  significant  effect  of  symbol  complexity  and  symbol 
on  accuracy  and  response  time  and  for  both  measures  combined.  However,  there  was  no 
significant  effect  of  background  or  interaction  between  background  and  symbol  complexity. 
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Figure  10:  Counting  accuracy  and  response  time  as  a  function  of  background  and  symbol 

complexity. 
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Table  6:  Summary  of  repeated  measures  analyses  of  variance  for  accuracy,  response  time,  and  a 

multivariate  analysis  of  both  measures. 


Source 

Univariate  / 
multivariate 
degrees  of  freedom 

F  values  (p  <  0.01) 

Accuracy 

Response  time 

Multi-variate 

Symbol  complexity 

1,8/ 2,7 

14.4 

143.9 

92.5 

Symbol 

8,64/16,126 

6.2 

42.6 

13.8 

Background  by 
symbol 

8,64/16,126 

n.s. 

3.9 

3.1 

Symbol  complexity 
by  symbol 

8,64/16,126 

n.s. 

24.5 

9.9 

There  was  also  a  significant  effect  of  symbol  shape  and  a  significant  interaction  between  symbol 
shape  and  both  background  and  symbol  complexity.  The  overall  effect  of  symbol  shape  was 
somewhat  different  than  for  Experiment  1.  A  post-hoc  analysis  of  symbol  shape  showed  that 
response  time  was  significantly  poorer  with  the  subsurface  symbols  compared  to  the  air  and 
surface  symbols  (except  for  the  air  unknown)  (Figure  11).  A  similar,  but  weaker,  effect  was  also 
found  with  the  accuracy  data. 


Response  time  (seconds) 

5  K>  ON  OO  © 

- 1 - 1 - 1 - 1 - 1 

III 

III 

III 

F 

riendly  Ftostile  Unknown  Friendly  Flostile  Unknown  Friendly  Hostile  Unknown 

Air  Surface  Subsurface 

Figure  11:  Counting  response  time  as  a  function  of  symbol  type. 

An  examination  of  the  interactions  between  symbol  shape  and  background  and  symbol  shape  and 
complexity  indicated  that  most  of  it  was  due  to  poorer  performance  with  the  subsurface  symbols 
in  the  complex  symbol  condition  (Figure  12).  As  well,  the  addition  of  the  complex  symbols 
appeared  to  eliminate  the  difference  in  performance  between  the  surface  and  air  symbols. 
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Figure  12:  Response  time  as  a  function  of  background  and  symbol  complexity  for  each  warfare 

area.  The  results  for  accuracy  were  similar. 

To  evaluate  these  effects  in  more  detail,  a  separate  analysis  was  carried  out  for  each  warfare  area 
(Table  7).  It  supported  the  effects  observed  in  Figure  12.  With  the  air  symbols,  there  was  no 
effect  of  either  background  or  symbol  complexity  on  accuracy,  response  time,  or  the  two 
measures  combined.  In  contrast,  performance  was  significantly  poorer  with  the  surface  and 
subsurface  complex  symbols  compared  to  the  basic  symbols.  As  well,  accuracy  was  significantly 
lower  against  the  complex  background  with  the  subsurface  symbols.  The  significant  interaction 
between  background  and  symbol  for  response  time  with  the  surface  symbols  is  shown  in 
Figure  13.  As  can  be  seen,  the  basic  hostile  surface  symbol  was  adversely  affected  by  the  map 
background  relative  to  the  friendly  and  unknown  symbols. 
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Table  7:  Summary  of  repeated  measures  analyses  of  variance  for  accuracy,  response  time,  and  a 
multivariate  analysis  of  both  measures  for  the  surface  and  subsurface  symbols. 


Source 

Univariate  / 
multivariate  degrees  of 
freedom 

Warfare 

area 

F  values  (p  value  < 

0.01) 

Accuracy 

Response 

time 

Multi¬ 

variate 

Background 

1,8/ 2,7 

Surface 

n.s. 

n.s. 

n.s. 

Subsurface 

20.0 

17.1 

Symbol 

1,8/ 2,7 

Surface 

12.3 

194.8 

85.8 

complexity 

Subsurface 

35.1 

202.5 

148.7 

Background  by 

8,64/16,126 

Surface 

09.5 

symbol 

Subsurface 

n.s 

n.s 

n.s 

-e 

a 

o 

41 
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Figure  13:  Response  time  as  a  function  of  background  and  symbol  complexity  for  the  surface  ID 

symbols  in  the  counting  task. 


Discussion 

Experiment  2  looked  at  the  impact  of  adding  icons  on  the  detection  and  discrimination  of  the 
basic  symbols.  In  most  cases  if  the  icon  was  presented  on  the  symbol,  its  presence  had  relatively 
little  impact  on  the  participant’s  ability  to  count  the  number  of  instances  of  a  specified  symbol  on 
the  display.  Poorer  performance  primarily  occurred  with  the  subsurface  icons  and  was  probably 
due  to  the  inclusion  of  the  mine  and  decoy  icons  in  that  set.  They  were  not  presented  against  the 
basic  symbol.  Although  the  colour  was  the  same,  participants  had  to  search  for  up  to  three 
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different  shapes  on  a  given  trial.  There  was  also  some  evidence  (Figure  12)  that  the  visibility  of 
the  surface  symbols  was  poorer  with  the  complex  symbols.  Without  the  icons,  the  surface 
symbols  were  usually  significantly  more  discriminable  than  the  air  and  subsurface  symbols. 
When  the  icons  were  present  this  was  no  longer  the  case.  More  research  is  needed  to  understand 
why  this  occurred.  It  could  be  due  to  the  presence  of  similar  icons  against  the  different  shapes  or 
to  a  reduction  in  the  colour  coded  area.  Whatever  the  reason,  it  does  suggest  that  any 
effort  to  improve  discrimination  amongst  the  basic  symbol  shapes  could  be  negated  by  the 
addition  of  icons. 

Presence  of  a  complex  background 

As  discussed  in  the  introduction,  the  addition  of  background  information  is  becoming  more 
common  on  tactical  displays.  The  results  of  Experiment  2  suggest  that  adding  such  information 
may  have  unexpected  consequences  even  when  existing  guidelines  are  followed.  The  major 
impact  was  with  the  subsurface  symbols.  Performance  was  significantly  degraded  with  the 
addition  of  a  complex  background.  Although  results  for  individual  icons  are  not  available,  it  is 
likely  that  the  poorer  performance  was  primarily  due  to  the  inclusion  of  the  mine  and  the  decoy. 
Even  with  the  thicker  outline,  subjectively,  they  were  not  as  easy  to  find  on  the  display.  It  may  be 
that  they  were  more  easily  confused  with  the  different  objects  in  the  background.  Another 
possibility  is  the  fact  that  these  shapes  include  critical  elements  with  relatively  high  spatial 
frequencies.  The  minimum  contrast  for  detecting  an  object  increases  rapidly  as  its  spatial 
frequency  increases  above  6  cycles  per  degree  (Campbell  and  Robson  1968,  McFadden  1994). 
These  results  also  suggest  that  outline  symbols  such  as  the  NTDS  symbol  set  would  suffer 
substantially  as  the  complexity  of  the  background  increased. 

One  way  of  improving  the  visibility  of  complex  icons  such  as  the  mine  and  decoy  is  to  make 
them  either  black  or  white,  depending  on  the  average  luminance  of  the  background.  This  is  the 
practice  followed  in  the  specification  of  colours  and  symbols  for  electronic  charts  (International 
Hydrographic  Organization  1997).  It  recommends  the  use  of  simple  geometric  shapes,  which 
have  a  relatively  low  spatial  frequency,  for  critical  information.  More  complex  symbols  are 
usually  black  or  white  to  maximize  their  contrast  and  legibility  given  that  they  have  a  lot  of  high 
spatial  frequencies.  This  would  restrict  the  use  of  icons  without  a  frame  to  objects  that  did  not 
have  an  ID.  Unfortunately,  whether  or  not  an  object  has  an  ID  is  often  a  function  of  the  taxonomy 
that  is  being  used  to  organize  the  symbology  (Harrison  2006).  For  example,  in  the  taxonomy  used 
for  MIL-STD-2525B,  ID  is  defined  at  a  very  high  level.  Thus,  even  inanimate  objects  such  as  oil 
platforms  are  given  an  ID. 

The  current  study  is  only  a  start  at  evaluating  the  impact  of  complex  backgrounds.  Current 
guidelines  usually  recommend  minimizing  the  use  of  colour  and  patterns  (Osga  and  Kellmeyer 
2000,  Unger  Campbell  2004).  However,  there  is  very  little  guidance  on  foreground  versus 
background  colours  or  the  interaction  between  different  categories  of  information.  These  results 
suggest  that  properly  designed  symbols  are  relatively  resistant  to  the  addition  of  colour  coded 
backgrounds  and  other  static  information.  However,  the  results  for  the  hostile  surface  symbols 
(Figure  13)  provide  a  caveat  to  that  conclusion.  Further  research  is  required  to  better  understand 
the  factors  that  lead  to  interactions  between  different  elements  in  a  complex  display.  The 
background  used  in  this  study  was  a  typical  electronic  chart  background.  The  background  colours 
conform  to  the  standard  (International  Hydrographic  Organization  1997)  which  requires  the  use 
of  desaturated  background  colours.  Other  static  information  and  non  critical  information  are 
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coded  in  black  or  grey  to  reduce  their  interference  with  more  critical  symbology.  Thus,  it  is  very 
important  that  these  results  not  be  used  to  justify  the  use  of  backgrounds  that  do  not  conform  to 
these  guidelines.  Even  with  good  design,  the  critical  nature  of  information  on  a  tactical  display 
makes  it  imperative  that  operators  have  the  option  of  removing  all  non  critical  background 
information  and  dimming  critical  contextual  information. 
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Experiment  3  -  Icon  discrimination 


Method 

Experiment  3  assessed  participants’  ability  to  discriminate  the  different  platform  icons.  Since  the 
basic  shape  delineated  warfare  area,  only  discrimination  of  icons  within  each  platform  area  was 
assessed.  To  avoid  interactions  with  symbol  shape  and  colour,  the  icons  were  always  presented 
against  the  hostile  symbol  shapes. 

Participants 

The  same  participants  that  completed  Experiment  2  took  part  in  Experiment  3. 

Conditions 

Experiment  3  was  a  three  (air,  surface,  and  subsurface)  by  ten  (platform  icon)  design.  The 
participants  searched  for  a  specific  platform  icon  in  a  background  composed  of  the  9  other 
platform  icons  for  that  warfare  area.  Participants  completed  30  runs,  one  for  each  platform  icon. 
The  participants  completed  all  the  runs  for  one  warfare  area  before  continuing  on  with  the  next 
set.  The  order  in  which  warfare  areas  were  presented  was  randomized  across  participants  and  the 
order  in  which  icons  were  presented  was  randomized  across  participants. 


Apparatus 

The  apparatus  was  identical  to  Experiment  2.  However,  a  sheet  showing  Table  4  was  located  on  a 
document  holder  to  the  right  of  the  monitor. 

Stimuli 

The  stimuli  for  Experiment  3  were  the  30  hostile  symbols  shown  in  Table  4. 

Display 

The  display  was  identical  to  Experiment  2  with  a  few  exceptions.  The  symbols  were  only 
presented  against  the  monochrome  grey  background  and  only  the  hostile  symbols  were  used.  The 
target  symbol  was  always  one  of  the  thirty  symbols  shown  in  Table  4  and  the  distractors  were  the 
other  platform  types  for  that  warfare  area.  Thus,  if  the  target  was  the  helicopter,  the  distractors 
would  be  the  other  nine  air  platforms. 

Task 

The  participants’  task  was  identical  to  Experiment  2  except  that  they  were  required  to  count  the 
number  of  occurrences  of  a  specific  platform  type. 
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Procedure 


The  procedure  was  identical  to  Experiment  2.  Experiment  3  was  carried  out  following  completion 
of  Experiment  2. 

Statistical  analysis 

The  data  preparation  and  basic  statistical  analysis  were  identical  to  Experiment  1. 


Results 

Experiment  3  assessed  how  discriminable  the  10  icons  in  each  warfare  area  were  from  each  other. 
Participants  searched  for  a  specific  icon  or  platform  type  in  a  background  composed  of  the 
remaining  icons  for  that  warfare  area.  All  of  the  icons  on  a  given  display  appeared  against  the 
same  basic  symbol  shape  to  avoid  having  to  search  across  different  coloured  symbols.  There  was 
a  significant  difference  in  overall  response  time  across  warfare  areas,  F(2,16)  =  23.4,  with  the 
post  hoc  analysis  indicating  that  the  time  to  count  the  surface  symbols  was  significantly  slower 
(1 1.0  seconds  versus  10.0  and  9.0  for  the  air  and  subsurface  warfare  area  respectively).  However, 
there  was  no  significant  difference  in  accuracy.  Since  the  individual  icons  were  not  comparable 
across  warfare  areas,  a  separate  analysis  was  carried  out  for  each  warfare  area.  For  all  three 
warfare  areas,  there  was  a  significant  effect  of  icon  (Table  8)  for  accuracy,  response  time  and  the 
two  measures  combined.  As  shown  in  Figure  14,  count  times  were  usually  fastest  for  the  basic 
symbols.  The  only  icons  with  similar  response  times  were  the  civilian  fishing  trawler  from  the 
surface  icons,  the  mine  and  the  decoy  from  the  subsurface  icons,  and  the  civilian  aircraft  and 
missile  from  the  air  icons.  The  results  in  Figure  14  were  supported  by  the  post-hoc  analysis.  The 
basic  symbol  was  located  significantly  faster  than  all  the  complex  symbols  and  in  turn  the  civilian 
aircraft  and  missile  were  located  significantly  faster  than  the  other  air  icons.  With  the  surface 
symbols  the  basic  symbol  and  the  civilian  fishing  trawler  were  located  significantly  faster  than 
the  remaining  surface  icons.  Finally  the  basic  symbol,  decoy  and  mine  were  located  significantly 
faster  than  the  remaining  subsurface  icons.  The  fastest  response  times  were  similar  to  those  for 
the  basic  symbol  in  Experiment  1.  Accuracy  results  were  similar  although  the  differences 
between  specific  icons  were  not  significant  in  most  cases. 

Table  8:  Summary  of  repeated  measures  analyses  of  variance  for  accuracy,  response  time,  and 
efficiency  for  the  air,  surface  and  subsurface  icons. 


Source 

Univariate  / 
multivariate 
degrees  of  freedom 

Warfare 

area 

F  values  (p  <  0.01) 

Accuracy 

Response 

time 

Multi¬ 

variate 

Air 

4.5 

56.4 

17.2 

Icon 

9,72/18,142 

Surface 

8.6 

207.3 

38.0 

Subsurface 

3.2 

61.9 

17.9 
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Figure  14:  Counting  response  time  for  each  of  the  complex  symbols  associated  with  the  air, 

surface  and  subsurface  warfare  areas. 


Discussion 

Experiments  3  examined  the  discriminability  of  the  different  icons  using  a  counting  task.  On 
average,  counting  performance  with  the  icons  was  similar  to  what  was  found  with  the 
monochrome  2525B  symbols.  This  suggests  that  most  of  the  shapes  were  not  distinctive  enough 
to  support  pre-attentive  processing.  However,  some  of  the  complex  symbols  were  located  as 
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rapidly  as  the  colour  coded  2525B  surface  symbols  in  Experiment  1.  Thus,  it  is  possible  to  create 
perceptually  distinct  icons.  In  most  cases,  the  best  icons  differed  in  colour  (basic  symbol  or  no 
icon,  missile,  civilian  aircraft,  and  tanker)  or  basic  shape  (decoy  and  mine)  from  the  remaining 
icons  or  symbols.  As  in  Experiment  1,  the  value  of  this  coding  seemed  to  depend  on  its 
uniqueness.  The  speed  and  accuracy  with  which  the  missile  and  aircraft  were  detected  was 
somewhat  poorer  than  the  civilian  tanker  which  was  the  only  light  coloured  icon  in  the  set  of 
surface  icons.  Overall  these  results  support  the  possibility  that  icons  could  be  designed  that  would 
allow  operators  to  scan  the  display  for  operationally  significant  platforms.  However,  the  number 
of  such  icons  that  could  be  generated  is  probably  relatively  small.  For  example,  the  use  of  a 
combination  of  shape  and  alphanumeric  icons  did  not  seem  to  reduce  the  search  space  for  the  air 
icons.  These  results  also  suggest  that  improper  use  of  these  perceptually  distinct  features  in 
coding  icons  could  have  a  negative  impact  on  performance.  In  MIL-STD-2525B  the  general 
practice  is  for  military  platforms  to  be  black,  civilian  platforms  white,  and  missiles  yellow.  This 
has  the  advantage  of  reducing  the  search  space  as  described  above.  However,  since  there  are 
usually  more  military  than  civilian  tracks,  it  does  not  necessarily  reduce  the  space  a  significant 
amount.  Moreover,  the  fact  that  counting  performance  for  the  white  civilian  aircraft  and  yellow 
missile  were  slower  than  for  the  civilian  trawler  suggests  that  the  yellow  and  white  were  not  that 
discriminable  and  this  coding  decision  could  lead  to  the  missiles  being  lumped  with  the  civilian 
platforms.  There  is  also  the  possibility  that  the  use  of  perceptually  distinct  icons  would  interfere 
with  grouping  the  symbols  on  the  basis  of  ID  or  warfare  area  as  was  suggested  in  Experiment  2 
for  the  surface  symbols.  However,  further  study  is  required  to  provide  a  definitive  answer 
on  these  issues. 
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Experiment  4  -  Icon  classification 


Method 

Experiment  4  examined  participants’  ability  to  quickly  and  accurately  associate  a  label  with  each 
of  the  icons.  The  icon  shape  should  either  intuitively  suggest  the  platform  type  being  represented 
or  operators  should  be  able  to  easily  learn  the  platform  type  represented  by  a  particular  icon. 


Participants 

The  participants  were  identical  to  Experiments  2  and  3. 


Conditions 

Experiment  4  was  a  three  (air,  surface,  and  subsurface)  by  ten  (platform  icon)  design.  Participants 
searched  for  one  of  the  nine  basic  2525B  symbol  shapes  and  then  identified  the  platform  icon  of 
the  target  symbol.  The  target  symbol  was  presented  in  a  background  composed  of  a  randomly 
selected  subset  of  the  remaining  80  symbols  (eight  basic  shapes  by  ten  platform  types).  The  basic 
symbol  shape  was  constant  across  a  run,  but  the  platform  icon  varied  randomly  from  trial  to  trial. 
Participants  completed  nine  runs  composed  of  10  practice  (one  example  of  each  platform  type) 
and  30  test  trials.  The  order  in  which  the  basic  symbol  shapes  were  presented  was  randomized 
across  participants. 


Apparatus 

The  apparatus  was  identical  to  Experiments  2  and  3.  However  for  Experiment  4,  participants  were 
assigned  one  of  three  different  keyboards2.  The  participant’s  assigned  keyboard  was  attached  to 
the  computer  at  the  beginning  of  each  of  his  or  her  sessions.  Each  keyboard  was  modified  by 
affixing  the  keys  with  labels  showing  the  terms  listed  in  Table  4.  The  labels  were  printed 
in  1 1  point  Times  Roman  font  for  a  nominal  visual  angle  of  about  17  to  20  minutes  of  arc.  They 
were  placed  on  the  10  number  keys,  and  the  alphabetic  keys  in  the  second  and  third  rows  plus  the 
Labels  for  each  warfare  area  were  placed  on  different  rows  that  were  also  labelled  with  the 
warfare  area.  Three  different  orders  for  the  location  of  the  warfare  area  (first,  second,  or  third 
row)  and  the  placement  of  the  labels  within  a  row  (see  Table  A2)  were  generated  to  control  for 
bias  and  the  different  orders  were  affixed  to  the  different  keyboards.  A  sheet  showing 
Table  A1  was  located  on  a  document  holder  to  the  right  of  the  monitor. 

Stimuli 

The  stimuli  used  in  Experiment  4  were  the  90  symbols  used  in  Experiment  2.  During  the  training 
sessions  the  symbols  were  presented  at  a  visual  angle  of  approximately  2  degrees  of  arc  at  a 
viewing  distance  of  60  cm  to  ensure  that  the  details  of  each  icon  were  clearly  visible. 

2  The  modified  keyboards  were  used  for  Experiments  2  and  3  as  well.  However,  participants  used  the 
number  pad  for  those  experiments. 
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Display 


Two  different  displays  were  used  in  Experiment  4  -  one  for  the  first  four  training  sessions  and  a 
second  for  the  final  training  session  and  the  experimental  session.  The  display  in  the  experimental 
session  was  identical  to  the  complex  symbol  condition  in  Experiment  2  except  that  there  was 
always  one  target  symbol  and  49  distractor  symbols.  For  the  training  sessions,  only  a  single 
symbol  was  presented  on  each  trial.  The  monochrome  grey  background  presented  in  Experiment 
2  was  used  in  both  the  training  and  test  sessions. 

Task 

During  the  training  phase  of  Experiment  4,  participants  were  shown  a  single  symbol  on  each  trial. 
They  were  required  to  press  the  key  containing  the  label  associated  with  that  symbol.  In  the 
experimental  session,  participants  were  required  to  locate  one  of  the  9  basic  symbol  shapes  and 
then  identify  the  icon  superimposed  on  that  shape.  The  participant  indicated  the  platform  by 
pressing  the  key  labelled  with  the  identifier  that  he  or  she  had  learned  to  associate  with  that  icon 
during  the  training  sessions.  Participants  always  had  Table  A1  available  to  assist  them  if  they 
forgot  the  keyboard  label  associated  with  an  icon. 


Procedure 

The  basic  procedure  was  identical  to  the  previous  experiments.  However,  the  training  sessions 
were  carried  out  in  parallel  to  Experiments  2  and  3.  During  each  training  session,  participants 
completed  9  runs  of  ten  trials  each.  On  each  trial  within  a  run,  they  were  shown  a  different 
complex  symbol  from  the  set  of  ten  complex  symbols  associated  with  a  basic  shape  and  were 
required  to  select  the  appropriate  label  for  the  icon  within  the  complex  symbol  on  the  keyboard. 
During  the  first  three  training  sessions,  the  three  ID  symbols  associated  with  one  warfare  area 
were  completed  before  presenting  the  ID  symbols  from  a  second  warfare  area.  The  order  in  which 
the  warfare  areas  were  presented  was  varied  across  days.  During  the  fourth  training  session,  the 
basic  symbol  shape  randomly  varied  from  run  to  run.  During  session  5,  the  participants  completed 
a  practice  session  that  was  identical  to  the  test  session,  except  that  it  had  fewer  trials,  followed  by 
the  test  session.  No  immediate  feedback  was  given  during  either  the  training  or  the  test  sessions 
in  Experiment  4  because  of  limitations  in  the  experimental  control  software.  However, 
participants  were  given  feedback  on  their  overall  accuracy  after  each  training  session. 


Statistical  analysis 

Data  preparation  for  response  times  was  identical  to  the  previous  experiments.  However,  no 
multi-variate  analysis  was  carried  out  as  accuracy  was  close  to  100%  across  all  conditions. 


Results 

In  Experiment  4,  participants  searched  for  a  basic  symbol  shape  and  then  identified  the  icon 
associated  with  the  shape.  Their  responses  and  response  times  were  recorded.  Prior  to  this 
experiment,  participants  completed  five  training  sessions  to  learn  to  associate  each  icon  with  a 
name  and  also  the  location  of  those  names  on  the  keyboard.  All  the  participants  averaged  better 
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than  95%  correct  and  average  response  times  dropped  from  4  seconds  to  2.2  seconds  across  the 
five  training  sessions.  Over  the  last  two  training  sessions,  response  time  ranged  from  1.2 
to  3.6  seconds  across  participants. 

A  between/within  ANOVA  showed  no  effect  of  keyboard  or  icon.  There  was  a  small,  but 
significant  interaction  (F(58,174)  =  1.7).  The  fastest  response  times  on  each  keyboard  tended  to 
be  associated  with  those  icons  that  were  at  the  ends  of  the  keyboard.  Mean  response  times  ranged 
from  2.1  to  2.4  seconds  across  the  three  keyboards. 

Collapsing  across  keyboards,  response  time  did  vary  significantly  (F(8,27)  =  6.2)  across  the  icons 
in  the  training  sessions.  Flowever,  there  were  no  significant  differences  between  specific  icons  in 
the  post-hoc  analysis.  This  is  not  surprising  given  the  variability  in  response  times  as  a  function 
of  keyboard  location  for  specific  icons.  Response  times  ranged  from  1.7  seconds  for  the 
submarine  station  to  4  seconds  for  the  rotary  wing  tanker  icon. 

Since  observers  had  to  search  for  the  basic  symbol  shape,  an  overall  analysis  of  symbol  shape 
was  carried  out  as  well  as  an  overall  analysis  of  icon  shape  for  the  data  in  the  test  sessions.  In 
addition,  icon  shape  was  analysed  as  a  function  of  warfare  area  since  we  were  primarily  interested 
in  the  discrimination  of  the  complex  symbols  within  each  area.  As  in  the  training  sessions, 
accuracy  was  quite  high.  It  ranged  from  95  to  99%  correct  across  symbol  shape  and  from  9 1  to 
1 00%  across  icon  shape.  These  differences  were  not  significant. 

Analysis  of  the  response  times  did  show  an  overall  effect  of  symbol  (F(8,64)  =  1 1.2)  with  the  post 
hoc  indicating  that  responses  to  the  surface  friendly  were  significantly  faster  than  to  the  air  and 
subsurface  hostile  and  unknown.  There  was  also  a  significant  effect  of  icon  (F9,72)  =  7.5,  0.01), 
but  the  post  hoc  analysis  showed  no  significant  differences  amongst  individual  icons.  The 
analysis  of  icons  within  a  warfare  area  showed  a  significant  effect  of  icon  for  all  three  warfare 
areas  (air:  F(9,72)  =  8.6,  surface:  F(9,72)  =  3.1,  subsurface:  F(9,72)  =  5.5).  As  indicated  in  Figure 
15,  the  post  hoc  analysis  showed  that  rotary  and  fixed  wing  tanker  icons  took  significantly  longer 
to  identify  than  the  fixed  wing  and  rotary  wing  icons.  For  the  subsurface  icons,  the  post  hoc 
analysis  indicated  that  responses  to  other-sub  were  significantly  slower  than  responses  to  the 
torpedo  icon.  Standard  errors  are  somewhat  larger  than  in  the  other  experiments.  This  is  probably 
due  to  variability  in  response  times  across  keyboards  and  across  participants. 
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Figure  15:  Response  time  for  each  of  the  icons  associated  with  the  air,  surface,  and  subsurface 
warfare  areas  in  the  last  two  sessions  of  the  training  runs  and  in  the  test  session.  The  icons  for 
each  warfare  area  appeared  against  the  three  ID  symbols. 
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Discussion 


Experiment  4  examined  the  intuitiveness  of  the  different  icons  using  an  identification  task. 
Participants  search  for  a  basic  symbol  shape  and  then  identified  the  icon  presented  in  or  in  place 
of  the  shape.  Although  many  of  the  icons  were  difficult  to  discriminate,  participants  were  able  to 
identify  them  accurately  and  relatively  quickly  in  Experiment  4.  Participants  identified  an  icon  by 
pressing  the  key  labelled  with  the  name  of  the  icon.  It  is  recognized  that  this  was  a  somewhat 
inefficient  method  of  collecting  responses.  It  was  unlikely  that  participant  would  memorize  the 
location  of  all  the  labels  within  a  row  in  the  training  time  provided.  Thus,  response  times  reflect 
both  the  time  to  locate  the  icon  label  as  well  as  the  time  to  identify  the  icon.  However,  it  is 
possible  to  get  some  estimate  of  the  time  required  to  locate  the  label  from  the  initial  training 
sessions.  Those  results  suggest  that  it  took  about  2  seconds  to  locate  the  label  on  average. 
Symbols  with  longer  response  times  were  either  less  intuitive  or  more  difficult  to  discriminate. 
Since  the  symbols  were  relatively  large,  it  is  unlikely  the  longer  times  were  due  to  legibility. 

The  response  times  in  the  test  session  reflect  the  above  factors  plus  the  time  to  locate  the  basic 
symbol  as  well  as  legibility  problems  due  to  the  reduction  in  size  of  the  complex  symbol.  Since 
the  response  times  were  averaged  over  the  three  ID  shapes  in  each  warfare  area, 
differences  in  the  increase  in  response  time  across  the  icons  within  a  warfare  area  should  be  due 
to  differences  in  discriminability. 

Taking  the  above  corrections  into  account,  the  longest  identification  times  were  found  with  the  air 
alphanumeric  icons.  Interestingly,  participants  did  not  have  a  similar  problem  with  the  surface 
alphanumeric  icons.  One  possibility  is  the  use  of  the  ‘A’  and  ‘K’  by  themselves  and  with  the 
rotary  wing  symbol  made  interpreting  those  icons  more  difficult.  That  would  not  explain  the  long 
response  time  for  the  fixed  wing  reconnaissance  icon.  An  examination  of  Table  A2  suggests 
another  possibility;  the  labels  for  these  icons  were  relatively  similar.  They  all  had  either  F.W.  or 
R.W.  Thus  at  least  part  of  the  increased  response  time  probably  resulted  from  the  choice  of  labels. 

Best  performance  overall  occurred  with  the  surface  icons.  Average  response  time  tended  to  be 
less  than  four  seconds  and  there  was  no  difference  across  icons.  The  use  of  relatively  distinct 
icons  and  relatively  equal  numbers  of  alphabetic  and  shape  icons  possibly  compensated  for  any 
reduction  in  legibility  with  reduced  size.  After  the  label  issue  has  been  taken  into  account,  the 
same  could  probably  be  said  for  the  air  icons.  The  exceptions  are  the  fixed  wing  and  rotary  wing 
tankers.  The  most  likely  reason  was  the  use  of  the  letter  K  for  tanker.  All  the  other  alphabetic 
icons  used  the  first  letter  of  the  platform  -  a  more  intuitive  choice. 

The  largest  increase  in  performance  between  the  training  and  test  runs  occurred  with  the 
subsurface  icons.  Icons  were  used  for  most  of  the  platform  types  and  their  shapes  were  similar. 
This  potentially  made  the  different  icons  less  discriminable  when  they  were  decreased  in  size  in 
the  test  conditions.  The  differences  in  response  time  between  the  training  and  test  runs  for  the 
mine  and  decoy  probably  occurred  for  a  different  reason.  The  mine  and  decoy  were  very  distinct, 
but  only  if  the  participant  knew  that  they  were  looking  for  them.  As  in  Experiment  2,  the  symbol 
that  participants  were  searching  for  could  have  any  one  of  three  shapes.  Thus,  the  results  with  the 
mine  and  decoy  substantiate  the  explanation  for  the  subsurface  results  for  Experiment  2. 

The  above  observations  are  tentative  at  best  given  the  relative  crudeness  of  the  response  method 
and  the  use  of  relatively  naive  participants.  Overall,  they  do  suggest  that  a  relatively  small  set  of 
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carefully  constructed  icons  can  be  accurately  interpreted  relatively  quickly,  but  that 
discrimination  and  identification  would  likely  decrease  rapidly  if  the  set  size  increased 
substantively.  Thus,  it  will  be  important  to  work  with  operators  to  determine  the  kinds  of  platform 
information  that  should  be  represented  with  icons  and  what  kinds  of  platforms  are  likely  to  be 
encountered  in  different  scenarios.  That  information  could  then  be  used  to  optimize  the  legibility 
and  discriminability  of  the  most  relevant  icons. 
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General  discussion 


The  four  experiments  reported  in  this  paper  were  carried  out  to  improve  our  understanding  of  the 
relative  strengths  of  the  NTDS  and  2525B  symbol  sets  and  to  develop  some  guidance  on  the  use 
of  iconic  information  on  a  tactical  display.  The  results  indicate  that  the  effective  design  of 
complex  symbols  that  are  discriminable  and  intuitive  is  not  easy.  Considerably  more  systematic 
research  is  required  and  the  current  results  should  be  treated  as  preliminary  at  best. 


Design  of  the  basic  symbol  shapes 

The  shapes  of  the  NTDS  and  2525B  symbol  sets  were  designed  to  discriminate  the  symbols  along 
the  dimensions  of  warfare  area  and  ID.  The  results  for  Experiment  1  suggest  that  this  goal  was 
not  achieved.  Specifically,  the  different  warfare  areas  are  not  equally  discriminable  and  the  most 
critical  warfare  area,  air,  is  not  the  most  discriminable.  Moreover,  this  problem  is  worse  with  the 
2525B  symbols  where  the  symbols  are  the  same  size.  As  discussed  previously,  efforts  to  improve 
discrimination  along  the  warfare  dimension  (e.g.  by  using  colour)  may  have  undesirable  side 
effects.  Thus,  it  is  probably  better  to  look  at  ways  of  allowing  operators  to  customize  their 
displays  to  support  their  critical  tasks.  Even  here,  it  is  important  to  carefully  evaluate  proposed 
concepts.  For  example,  in  their  investigation  of  variable  coded  symbology,  Van  Orden  et  al 
(1999)  discovered  that  poor  performance  was  linked  with  the  use  of  both  white  and  grey  symbols 
on  the  same  display. 

Discrimination  is  especially  poor  with  the  neutral  symbol  proposed  in  MIL-STD-2525B.  As 
discussed  earlier,  it  was  impossible  to  discriminate  amongst  the  neutral  symbols  for  the  three 
warfare  areas.  A  possible  alternative,  proposed  in  draft  STANAG  4420,  was  a  notched  square. 
The  notch  was  at  the  bottom  for  air  platforms  and  the  top  for  subsurface  platforms.  The  U.S. 
Navy  implementation  Guide  (NAVSEA  Dahlgren  2006)  recommends  the  use  of  the  notched 
symbol,  but  only  when  the  basic  symbol  is  used  without  iconic  information.  If  operators  are 
simply  examining  specific  symbols,  the  icon  shape  is  a  useful  cue  about  the  track  warfare  area. 
However,  if  operators  are  scanning  a  display  looking  for  new  tracks  or  changes  in  tracks,  the 
results  for  Experiment  3  suggest  that  searching  for  icons  associated  with  a  specific  warfare  area  is 
often  slow  and  inefficient.  A  more  efficient  strategy  would  be  to  use  the  basic  symbol  shape  to 
reduce  the  size  of  the  search  space  and  then  examine  specific  icons  within  that  space. 
The  notched  symbols  are  better  for  this  purpose  than  the  simple  square  specified  in  MIL- 
STD-2525B. 

Use  of  complex  symbols 

The  primary  benefit  expected  from  providing  iconic  information  on  the  tactical  display  is  a 
reduction  in  the  requirement  to  select  a  symbol  in  order  to  get  amplifying  information  about  a 
specific  track  (NAVSPAWARSYSCOM  1991).  In  order  to  achieve  this  advantage,  the  icons  need 
to  be  legible,  intuitive,  and  discriminable.  Although  icons  are  widely  used,  both  in  electronic 
systems  and  in  the  environment,  there  is  little  evidence  that  these  criteria  are  ever  fully  met.  To 
start,  icons  are  rarely  as  intuitive  as  their  designers  expect  (Smallman,  St.  John,  Oonk,  and  Cowen 
2000).  Legibility  and  discriminability  are  usually  compromised  by  the  need  to  make  them 
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relatively  small  to  avoid  clutter  and  overlap.  Recent  research  has  suggested  that  icons  should 
subtend  at  least  45  minutes  of  arc  (Lindberg  and  Nasanen  2003).  With  tactical  displays,  there  is 
the  additional  constraint  that  the  addition  of  platform  information  should  not  impact  the  visibility 
of  the  other  information  encoded  in  the  symbol.  Experiment  2  addressed  this  final  requirement 
while  Experiments  3  and  4  primarily  examined  legibility  and  discriminability.  Since  the  icons 
represent  tactical  platforms  and  we  were  using  non-military  participants,  it  was  not 
possible  to  investigate  intuitiveness  directly  although  the  results  did  provide  some 
evidence  of  its  importance. 

Experiments  3  and  4  examined  the  discriminability  of  the  different  icons  using  a  counting  and  an 
identification  task  respectively.  As  discussed  previously,  the  results  suggest  that  it  is  possible  to 
design  icons  that  would  allow  operators  to  scan  the  display  for  operationally  significant 
platforms.  Elowever,  the  number  of  such  icons  that  could  be  generated  is  probably  relatively  small 
and  the  indiscriminate  use  of  these  perceptually  distinct  features  in  coding  icons  could  have  a 
negative  impact  on  performance. 

The  effective  location  of  the  mine  and  decoy  in  Experiment  3  suggests  that  the  use  of  icons 
without  a  frame  (not  superimposed  on  a  basic  symbol  shape),  would  be  another  method  for 
highlighting  operationally  critical  platforms.  Their  unique  shape  made  them  highly  conspicuous. 
Elowever,  the  results  for  Experiments  2  and  4  indicate  that  this  could  interfere  with  the  operator’s 
ability  to  monitor  all  of  the  tracks  for  a  particular  warfare  area  or  with  a  specific  ID.  Thus,  the 
decision  to  use  an  icon  without  a  frame  has  to  be  carefully  considered.  STANAG  4420  restricted 
the  use  of  icons  without  a  frame  to  objects  that  did  not  have  tactical  significance.  However,  this  is 
often  difficult  to  determine  and  could  be  risky.  Even  a  seemly  innocuous  pleasure  boat  can 
become  a  hostile  contact.  Moreover  using  unique  shapes  for  non  critical  tracks  would  be  at  odds 
with  highlighting  operationally  critical  platforms.  In  general,  it  is  possible  that  the 
effective  use  of  icons  without  frames  is  context  dependent.  Further  research  is  required  to 
determine  how  to  use  them  effectively. 

It  should  be  noted  that  the  above  recommendation  tends  to  be  at  odds  with  current  developments 
(Department  of  Defence  2006).  In  general,  there  seems  to  be  a  move  towards  the  use  of  larger 
numbers  of  icons  to  represent  the  known  characteristics  of  specific  platforms.  This  leads  to  very 
complex  icons  and  a  corresponding  decrease  in  legibility.  The  basis  for  this  decision  is  not  clear. 
It  may  be  that  even  if  only  a  small  percentage  of  the  icons  are  readily  identified,  average  response 
time  is  reduced  relative  to  selecting  or  hooking  each  track  and  reading  the  platform  information. 
Another  possibility  is  that  most  of  the  research  investigates  only  a  subset  of  the  icons  or  does  not 
employ  time-sensitive  tasks.  It  is  important  to  test  how  quickly  operators  locate  specific  icons 
under  operational  conditions  as  well  as  how  intuitive  the  icons  are.  Thus,  further  research  is 
needed  to  systematically  evaluate  the  operational  values  of  these  complex  icons. 


DRDC  Toronto  TR  2007-046 


45 


Recommendations 


Based  on  the  results  of  the  experiments  reported  in  this  paper,  it  is  recommended  that  the 
following  work  be  undertaken; 

1.  Implement  a  modified  form  of  the  2525B  symbol  set  for  representing  contacts  on  tactical 
displays  especially  if  complex  backgrounds  are  being  considered  or  if  is  desirable  to 
provide  iconic  information  about  platform  type.  Necessary  modifications  include: 

a.  Replacing  or  redesigning  the  current  neutral  symbol; 

b.  Outlining  the  symbols  in  a  solid  line  at  least  two  pixels  wide  to  ensure  their 
visibility  against  complex  backgrounds; 

c.  Restricting  the  use  of  icons  without  frames  to  objects  that  are  not  tactically 
significant. 

2.  Conduct  further  research  to  evaluate  proposed  methods  for  improving  discrimination 
between  air  and  subsurface  symbols.  These  studies  must  include  time  sensitive  tasks. 

3.  Conduct  further  research  to  evaluate  proposed  guidelines  for  complex  backgrounds  and 
symbols  that  will  minimize  the  impact  of  the  backgrounds  on  the  visibility  of  tactical 
symbols. 

4.  Conduct  analysis  and  research  to  develop  more  effective  symbol  taxonomies. 

5.  Conduct  analysis  and  research  to  develop  guidelines  for  icon  construction  that: 

a.  Maximize  differences  amongst  icons  that  are  likely  to  be  presented 
simultaneously; 

b.  Ensures  operationally  critical  icons  are  unique; 

c.  Minimizes  the  effect  of  the  icons  on  the  visibility  of  other  information 
represented  by  the  symbols. 
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Conclusion 


Four  experiments  were  carried  out  to  investigate  the  impact  of  moving  from  the  NTDS  symbol 
set  to  the  MIL-STD-2525B  symbol  set  for  representing  tracks  on  naval  tactical  displays. 
Experiment  1  assessed  the  visibility  of  the  basic  Naval  Tactical  Data  System  (NTDS)  and 
MIL-STD-2525B  (2525B)  tactical  symbols.  Experiments  2  through  4  measured  the 
discrimination  of  the  basic  symbols  with  iconic  information  added  and  discrimination  of  the  icons 
themselves.  In  addition,  Experiment  2  investigated  the  effect  of  presenting  the  basic  and  complex 
2525B  symbols  against  a  complex  background.  The  results  for  Experiment  1  indicated  that  the 
colour  coded  2525B  symbol  set  offered  no  advantage  over  a  well  designed  NTDS  symbol  set  if 
both  were  presented  against  a  plain  dark  grey  background.  However,  the  results  for  Experiment  2 
suggest  that  colour-coded  solid  shapes  outlined  in  black  are  probably  more  visible  than  colour 
coded  outline  shapes  when  presented  on  complex  backgrounds.  In  addition,  iconic  information 
can  be  added  more  easily  to  the  solid  shapes.  For  the  most  part,  adding  iconic  information  did  not 
affect  how  quickly  and  accurately  the  2525B  symbols  were  detected  except  when  the  icon  shape 
replaced  the  symbol  shape.  Accuracy  and  response  time  in  detecting  and  recognizing  individual 
icons  depended  on  their  complexity  and  their  uniqueness.  It  was  recommended  that  only  a  small 
number  of  highly  discriminable  icons  be  used  at  any  one  time.  The  use  of  icons  by  themselves 
should  be  restricted  to  non-tactical  information  and  probably  they  should  not  be  colour  coded.  It 
was  also  recommended  that  a  more  discriminable  method  of  representing  a  neutral  contact  be 
developed  and  tested.  The  MIL-STD-2525B  symbols  for  these  types  of  contacts  are  unsuitable  for 
use  on  electronic  displays  when  presented  at  the  recommended  visual  angle.  Further  research  is 
required  to  evaluate  proposed  methods  for  improving  discrimination  between  the  air  and 
subsurface  symbols  and  between  tactical  (foreground)  and  contextual  (background)  information. 
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Annex  A  Symbol  sheet  and  keyboard  layout 


Table  Al:  Symbol  sheet  used  in  Experiment  4 


Fixed  Wing 

Combatant 

A  F.W  Attack 

Carrier 

Nuc  Prop 

F.W  Tanker 

B  B  Battleship 

11  Con  Prop 

B  F.W  Recon 

B  D  Destroyer 

Other  Sub 

Rotary  Wing 

p  p  Frigate 

Sub  Station 

SS 

R.W  Attack 

Mine  War 
Ves 

WPN  Sub  Weapon 

R.W  Tanker 

|\/|§  Minesweeper 

Mine 

i 

A 

Air  Weapon 

Patrol 

<]  <|<j  Decoy 

F.W  Civil 

Civ  Fishing 

Torpedo 

0 

Air  No  Icon 

<2> 

O 

Surf  No  Icon 

O 

u 

Sub  No  Icon 

<3> 
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Table  A2:  Location  of  labels  on  the  three  keyboards  used  in  Experiment  4 


Keyboard  1 

F.W. 

Attack 

R.W. 

Attack 

Air 

No  Icon 

Aii- 

Weapon 

R.W. 

T  anker 

F.W. 

T  anker 

F.W. 

Recon 

Fixed 

Wing 

Rotaiy 

Wing 

F.W. 

Civil 

Battle 

ship 

Mine 

sweeper 

Com¬ 

batant 

Frigate 

Civ. 

Fishing 

Carrier 

Mine 

War. 

Vess. 

Surf 

No 

Icon 

Destr¬ 

oyer 

Patrol 

Sub 

Sub 

Station 

Torpedo 

Con 

Prop  Sub 

Mine 

Other 

Sub 

Decoy 

Nuc 

Prop 

Sub 

Sub 

Weapon 

Sub 

No 

Icon 

Keyboard  2 

Com¬ 

batant 

Frigate 

Civ. 

Fishing 

Battle 

ship 

Mine 

sweeper 

Destr¬ 

oyer 

Patrol 

Carrier 

Mine 

War. 

Vess. 

Surf 

No  Icon 

Nuc 

Prop 

Sub 

Sub 

Weapon 

Sub 

No  Icon 

Other 

Sub 

Decoy 

Con 

Prop 

Sub 

Mine 

Sub 

Sub 

Station 

Torpedo 

F.W. 

Tanker 

F.W. 

Recon 

Fixed 

Wing 

Rotary 

Wing 

F.W. 

Civil 

F.W. 

Attack 

R.W. 

Attack 

Air 

No 

Icon 

Aii- 

Weapon 

R.W. 

Tanker 

Keyboard  3 

Con 

Prop 

Sub 

Mine 

Sub 

Sub 

Station 

Torpedo 

Nuc 

Prop  Sub 

Sub 

Weapon 

Sub 

No 

Icon 

Other 

Sub 

Decoy 

Fixed 

Wing 

Rotary 

Wing 

F.W. 

Civil 

F.W. 

Tanker 

F.W. 

Recon 

Aii- 

Weapon 

R.W. 

Tanker 

F.W. 

Attack 

R.W. 

Attack 

Air 

No  Icon 

Carrier 

Mine 

War. 

Vess. 

Surf 

No 

Icon 

Destr¬ 

oyer 

Patrol 

Battle 

ship 

Mine 

sweeper 

Com¬ 

batant 

Frigate 

Civ. 

Fishing 

52 


DRDC  Toronto  TR  2007-046 


List  of  symbols/abbreviations/acronyms/initialisms 


CF 

Canadian  Forces 

CCS 

Command  and  Control  System 

COMDAT 

COMmand  Decision  Aiding  Technology 

DRDC 

Defence  Research  and  Development  Canada 

HMCCS 

Halifax  Class  Modernization  Command  and  Control  System 

ID 

identification  (especially  Standard  Identification) 

MIL-STD-2525B 

Military  Standard  2525B 

MSDF 

Multi  Source  Data  Fusion 

NATO 

North  Atlantic  Treaty  Organisation 

NTDS 

Naval  Tactical  Data  System 

OMI 

Operator-Machine  Interface 

STANAG 

STANdardisation  AGreement 

TDP 

Technology  Demonstrator  Programme 
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recommandations  pour  T  amelioration  de  la  discrimination  des  differents  symboles  utilises  dans 
les  zones  de  guerre  sont  foumies. 

L’un  des  avantages  du  jeu  de  symboles  2525B  est  d’offrir  la  possibilite  d’ajouter  a  la  forme  de 
base  du  symbole  de  Tinformation  sur  la  plate-forme  de  poursuite  utilisee.  Les  autres  experiences 
portaient  sur  revaluation  de  la  visibilite  des  symboles  de  base  lorsque  de  l’information  iconique 
est  ajoutee  et  sur  la  visibilite  des  icones  elles-memes.  L’ajout  d’information  iconique  n’avait  pas 
un  effet  important  sur  Tefficacite  avec  laquelle  les  symboles  de  base  etaient  localises,  sauf 
lorsque  la  forme  de  Ticone  rempla9ait  la  forme  du  symbole.  Le  rendement  en  termes  de 
localisation  et  de  reconnaissance  d’ icones  individuelles  dependait  de  leur  complexite  et  de  leur 
unicite.  II  est  recommande  de  n’utiliser  simultanement  qu’un  petit  nombre  d’icones  hautement 
discriminables.  L’utilisation  d’icones  sans  le  cadre  du  symbole  devrait  etre  limitee  uniquement  a 
P information  non  tactique  et  ces  icones  ne  devraient  probablement  pas  comporter  un 
codage  en  couleurs.  D’autres  recherches  sont  necessaires  afm  de  determiner  comment  mettre  en 
oeuvre  ces  recommandations. 


14.  KEYWORDS,  DESCRIPTORS  or  IDENTIFIERS  (Technically  meaningful  terms  or  short  phrases  that  characterize  a  document  and  could  be 
helpful  in  cataloguing  the  document.  They  should  be  selected  so  that  no  security  classification  is  required.  Identifiers,  such  as  equipment  model 
designation,  trade  name,  military  project  code  name,  geographic  location  may  also  be  included.  If  possible  keywords  should  be  selected  from  a 
published  thesaurus,  e.g.  Thesaurus  of  Engineering  and  Scientific  Terms  (TEST)  and  that  thesaurus  identified.  If  it  is  not  possible  to  select 
indexing  terms  which  are  Unclassified,  the  classification  of  each  should  be  indicated  as  with  the  title.) 

COMDAT;  Navy;  MIL-STD-2525B;  NTDS  symbols;  operator-machine  interface;  symbols; 
icons;  tactical  symbology;  multi-dimensional  scaling 
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